Encoder, decoder and corresponding method

Abstract

To provide a video coding mechanism.SOLUTION: A mechanism includes receiving a bitstream including a subpicture that includes a slice. A slice address is obtained. The slice address includes a subpicture level slice index of the slice when the slice is a rectangular slice. A slice of the subpicture is decoded on the basis of the slice address. The slice is transferred for display as part of a decoded video sequence.SELECTED DRAWING: Figure 5B

Description

[Technical Field]

[0001] [Technical field] FIELD This disclosure relates generally to video coding, and more particularly to coding sub-pictures of pictures in video coding. [Background technology]

[0002] The amount of video data required to render even a relatively short video can be considerable, which can create difficulties when the data is streamed or communicated across communication networks with limited bandwidth capacity. Therefore, video data is typically compressed before being communicated across modern telecommunications networks. Video size can also be an issue when the video is stored on a storage device, as memory resources may be limited. Video compression devices often use software and / or hardware at the source to code the video data before transmission or storage, thereby reducing the amount of data needed to represent a digital video image. The compressed data is then received at the destination by a video decompression device, which decodes the video data. Due to limited network resources and the ever-increasing demand for higher video quality, improved compression and decompression techniques that improve compression ratios with little or no sacrifice in image quality are desirable. Summary of the Invention

[0003] In an embodiment, the present disclosure includes a method implemented in a decoder, the method including: receiving, by a receiver of the decoder, a bitstream including a sub-picture including a slice; obtaining, by a processor, a slice address including a sub-picture-level slice index of the slice when the slice is a rectangular slice; and decoding, by the processor, the slice of the sub-picture based on the slice address. A picture can be partitioned into multiple slices for encoding. In some video coding systems, slices are addressed based on their position relative to the picture. Still other video coding systems use the concept of sub-pictures. Sub-pictures can be treated differently from other sub-pictures from a coding perspective. For example, sub-pictures can be extracted and displayed independently of other sub-pictures. In such cases, slice addresses generated based on picture position may not work properly because a significant number of potential slice addresses are omitted. Some video coding systems address this issue by dynamically rewriting slice headers upon request to change the slice address to support sub-picture extraction. Such a process can be resource intensive because it may occur every time a user requests to view a sub-picture. This disclosure includes slices addressed relative to the subpictures that contain them. For example, a slice header may include a subpicture identifier (ID) and the address of each slice included in the subpicture. Furthermore, a sequence parameter set (SPS) may include dimensions of the subpicture that can be referenced by the subpicture ID. Thus, when separate extraction of a subpicture is required, the slice header does not need to be rewritten. The slice header and SPS contain sufficient information to support positioning of slices within a subpicture for display.This example therefore increases coding efficiency and / or avoids redundant rewriting of slice headers, thus reducing the use of processor, memory and / or network resources in the encoder and / or decoder.

[0004] Optionally, in any of the above aspects, another implementation method of the aspect provides that the aspect further includes the steps of: obtaining, by a processor, a subpicture identifier (ID) of a subpicture that includes the slice; and locating, by the processor, the slice within the subpicture based on the slice address and the subpicture ID.

[0005] Optionally, in any of the above aspects, another implementation manner of the aspect provides that the bitstream further includes a slice header associated with the slice, and the slice address and sub-picture ID are obtained from the slice header.

[0006] Optionally, in any of the above aspects, another implementation of the aspect provides that when rect_slice_flag is equal to 1, the slice is a rectangular slice.

[0007] Optionally, in any of the above aspects, another implementation manner of the aspect provides that the slice address is obtained from a slice_address variable, and the subpicture ID is obtained from a slice_subpic_id variable.

[0008] Optionally, in any of the above aspects, another implementation manner of the aspect provides that the bitstream further includes a sequence parameter set (SPS), the SPS includes a description of the subpicture, and the subpicture ID references the description of the subpicture.

[0009] Optionally, in any of the above aspects, another implementation of the aspect provides that the description of the subpicture includes a width of the subpicture in units of coding tree blocks (CTBs), a height of the subpicture in units of CTBs, and an offset of the subpicture specified as the vertical position of the subpicture's top-left coding tree unit (CTU) and the horizontal position of the subpicture's top-left CTU.

[0010] In an embodiment, the present disclosure includes a method implemented in an encoder, the method including the steps of: partitioning, by a processor of the encoder, a subpicture into slices; determining, by the processor, slice addresses including subpicture-level slice indices for the slices when the slices are rectangular slices; encoding, by the processor, the slice addresses into a bitstream; encoding, by the processor, the slices of the subpicture into the bitstream; and storing, by a memory coupled to the processor, the bitstream for communication to a decoder. A picture can be partitioned into multiple slices for encoding. In some video coding systems, slices are addressed based on their position relative to the picture. Still other video coding systems use the concept of subpictures. Subpictures can be treated differently from other subpictures from a coding perspective. For example, subpictures can be extracted and displayed independently of other subpictures. In such cases, slice addresses generated based on picture position may not work properly because a significant number of potential slice addresses are omitted. Some video coding systems address this issue by dynamically rewriting slice headers upon request to change slice addresses to support subpicture extraction. This process can be resource-intensive because it must occur each time a user requests to view a subpicture. This disclosure includes slices addressed relative to the subpictures that contain them. For example, a slice header may include a subpicture identifier (ID) and the address of each slice included in the subpicture. Furthermore, a sequence parameter set (SPS) may include the dimensions of the subpicture, which may be referenced by the subpicture ID. Thus, slice headers do not need to be rewritten when separate extraction of a subpicture is requested.The slice header and SPS contain sufficient information to support positioning of the slice within the subpicture for display. Thus, this example increases coding efficiency and / or avoids redundant rewriting of slice headers, thus reducing the use of processor, memory, and / or network resources in the encoder and / or decoder.

[0011] Optionally, in any of the above aspects, another implementation method of the aspect provides that the method further includes a step of determining, by a processor, a subpicture ID of a subpicture that includes the slice, and a step of encoding, by the processor, the subpicture ID into a bitstream.

[0012] Optionally, in any of the above aspects, another implementation manner of the aspect is provided that the aspect further includes a step of encoding, by a processor, a slice header associated with the slice into the bitstream, wherein the slice address and subpicture ID are encoded in the slice header.

[0013] Optionally, in any of the above aspects, another implementation of the aspect provides that when rect_slice_flag is equal to 1, the slice is a rectangular slice.

[0014] Optionally, in any of the above aspects, another implementation of the aspect provides that the slice address is coded in a slice_address variable, and the subpicture ID is coded in a slice_subpic_id variable.

[0015] Optionally, in any of the above aspects, another implementation method of the aspect is provided that the aspect further includes a step of encoding, by a processor, the SPS into a bitstream, wherein the SPS includes a description of the subpicture and the subpicture ID references the description of the subpicture.

[0016] Optionally, in any of the above aspects, another implementation of the aspect provides that the subpicture description includes a subpicture width in units of CTBs, a subpicture height in units of CTBs, and a subpicture offset specified as the vertical position of the subpicture's top-left CTU and the horizontal position of the subpicture's top-left CTU.

[0017] In an embodiment, the present disclosure includes a video coding device including a processor, a receiver coupled to the processor, a memory coupled to the processor, and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform the method of any of the above aspects.

[0018] In an embodiment, the present disclosure includes a non-transitory computer-readable medium including a computer program product for use by a video coding device, the computer program product including computer-executable instructions stored on the non-transitory computer-readable medium that, when executed by a processor, cause the video coding device to perform the method of any of the above aspects.

[0019] In an embodiment, the present disclosure includes a decoder including receiving means for receiving a bitstream including a subpicture including a slice; obtaining means for obtaining a slice address including a subpicture level slice index of the slice when the slice is a rectangular slice; decoding means for decoding the slice of the subpicture based on the slice address; and transferring means for transferring the slice for display as part of a decoded video sequence.

[0020] Optionally, in any of the above aspects, another implementation manner of the aspect provides that the decoder is further configured to perform the method of any of the above aspects.

[0021] In an embodiment, the present disclosure includes an encoder including a partitioning means for partitioning a subpicture into slices, a determining means for determining a slice address including a slice index of a subpicture level of the slice when the slice is a rectangular slice, an encoding means for encoding the slice address into a bitstream and encoding the slice of the subpicture into a bitstream, and a storage means for storing the bitstream for communication to a decoder.

[0022] Optionally, in any of the above aspects, another implementation of the aspect provides that the encoder is further configured to perform the method of any of the above aspects.

[0023] Optionally, in any of the above aspects, another implementation of the aspect provides that the encoder is further configured to perform the method of any of the above aspects.

[0024] For purposes of clarity, any one of the above embodiments may be combined with any one or more of the other above embodiments to create new embodiments within the scope of the present disclosure.

[0025] These and other features will be more clearly understood from the following detailed description considered in conjunction with the accompanying drawings and claims. [Brief explanation of the drawings]

[0026] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, wherein like reference numerals represent like parts. [Figure 1] 1 is a flowchart of an exemplary method for coding a video signal. [Figure 2] 1 is a schematic diagram of an example coding and decoding (codec) system for video coding. [Figure 3] FIG. 1 is a schematic diagram illustrating an exemplary video encoder. [Figure 4] FIG. 1 is a schematic diagram illustrating an exemplary video decoder. [Figure 5A] FIG. 1 is a schematic diagram illustrating an exemplary picture partitioned into sub-pictures. [Figure 5B] FIG. 2 is a schematic diagram illustrating an exemplary subpicture partitioned into slices. [Figure 5C] FIG. 2 is a schematic diagram illustrating an exemplary slice partitioned into tiles. [Figure 5D] 1 is a schematic diagram illustrating an example slice partitioned into coding tree units (CTUs). [Figure 6] FIG. 1 is a schematic diagram illustrating an example of unidirectional inter prediction. [Figure 7] FIG. 1 is a schematic diagram illustrating an example of bidirectional inter prediction. [Figure 8] FIG. 10 is a schematic diagram illustrating an example of coding a current block based on candidate motion vectors from neighboring coded blocks. [Figure 9] FIG. 2 is a schematic diagram illustrating an exemplary pattern for determining a candidate list of motion vectors. [Figure 10] FIG. 2 is a block diagram illustrating an example in-loop filter. [Figure 11] FIG. 1 is a schematic diagram illustrating an example bitstream including coding tool parameters for supporting decoding sub-pictures of a picture. [Figure 12] 1 is a schematic diagram of an exemplary video coding device. [Figure 13] 1 is a flowchart of an exemplary method for encoding a video sequence into a bitstream by addressing slices relative to subpictures. [Figure 14] 1 is a flowchart of an exemplary method for decoding a video sequence from a bitstream by addressing slices relative to subpictures. [Figure 15]1 is a schematic diagram of an exemplary system for coding a video sequence of images in a bitstream when slices are addressed relative to sub-pictures; DETAILED DESCRIPTION OF THE INVENTION

[0027] First, while exemplary implementations of one or more embodiments are provided below, it should be understood that the disclosed systems and / or methods may be implemented using any number of technologies currently known or in existence. The present disclosure should in no way be limited to the exemplary implementations, drawings, and technologies set forth below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims, along with the full range of equivalents thereof.

[0028] The following abbreviations are used: Adaptive Loop Filter (ALF), Coding Tree Block (CTB), Coding Tree Unit (CTU), Coding Unit (CU), Coded Video Sequence (CVS), Joint Video Experts Team (JVET), Motion-Constrained Tile Set (MCTS), Maximum Transfer Unit (MTU), Network Abstraction Layer (NAL), Picture Order Count (POC), Raw Byte Sequence Payload (RBSP), Sample Adaptive Offset (SAO), Sequence Parameter Set (SPS), Temporal Motion Vector Prediction (TMVP), Versatile Video Coding (VVC), and Working Draft (WDR). WD) is used here.

[0029] Many video compression techniques can be used to reduce the size of video files with minimal data loss. For example, video compression techniques may include performing spatial (e.g., intra-picture) prediction and / or temporal (e.g., inter-picture) prediction to reduce or remove data redundancy in a video sequence. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks, which may also be referred to as tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are coded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded unidirectionally predicted (P) or bidirectionally predicted (B) slice of a picture may be coded using spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture may be referred to as a frame and / or an image, and a reference picture may be referred to as a reference frame and / or a reference image. Spatial or temporal prediction results in a prediction block that represents an image block. Residual data represents pixel differences between the original image block and the prediction block. Thus, inter-coded blocks are coded according to a motion vector that points to a block of reference samples that form the prediction block, and residual data that indicates the difference between the block being coded and the prediction block. Intra-coded blocks are coded according to an intra-coding mode and residual data. For further compression, the residual data may be transformed from the pixel domain to a transform domain, resulting in residual transform coefficients, which may be quantized. The quantized transform coefficients may first be arranged in a two-dimensional array. The quantized transform coefficients may then be scanned to generate a one-dimensional vector of transform coefficients.Entropy coding may be applied to achieve even greater compression, and such video compression techniques are described in more detail below.

[0030] To ensure that the encoded video can be accurately decoded, the video is encoded and decoded according to a corresponding video coding standard, including International Telecommunication Union (ITU) Standardization Sector (ITU-T) H.261, International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Motion Picture Experts Group (MPEG)-1 Part 2, Advanced Video Coding (AVC), also known as ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or ISO / IEC MPEG-4 Part 10, and High Efficiency Video Coding (HEVC), also known as ITU-T H.265 or MPEG-H Part 2. AVC includes extensions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding plus Depth (MVC+D), and three dimensional (3D) AVC (3D-AVC). HEVC includes extensions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D HEVC (3D-HEVC).The ITU-T and ISO / IEC Joint Video Working Group (JVET) has begun developing a video coding standard called Versatile Video Coding (VVC). VVC is contained in a Working Draft (WD), which includes JVET-M1001-v6, which provides an algorithm description, an encoder-side description of VVC WD, and reference software.

[0031] To code a video image, the image is first partitioned, and the partitions are coded into a bitstream. Various picture partitioning schemes are available. For example, an image can be partitioned into normal slices, dependent slices, tiles, and / or according to wavefront parallel processing (WPP). For simplicity, HEVC restricts encoders to only use normal slices, dependent slices, tiles, WPP, and combinations thereof when partitioning slices into groups of CTBs for video coding. Such partitioning can be applied to support Maximum Transfer Unit (MTU) size matching, parallel processing, and reduced end-to-end delay. The MTU indicates the maximum amount of data that can be transmitted in a single packet. If a packet payload exceeds the MTU, the payload is split into two packets through a process called fragmentation.

[0032] A normal slice, also simply referred to as a slice, is a partitioned portion of an image that can be rendered independently of other normal slices within the same picture, despite some interdependencies due to loop filtering operations. Each normal slice is encapsulated in its own Network Abstraction Layer (NAL) unit for transmission. Furthermore, intra-picture prediction (intra-sample prediction, motion information prediction, coding mode prediction) and entropy coding dependencies across slice boundaries may be disabled to support independent rendering. Such independent rendering supports parallelization. For example, normal slice-based parallelization uses minimal inter-processor or inter-core communication. However, because each normal slice is independent, each slice is associated with an individual slice header. The use of normal slices can incur substantial coding overhead due to the bit cost of a slice header for each slice and the lack of prediction across slice boundaries. Furthermore, normal slices may be used to support matching for MTU size requirements. Specifically, because regular slices can be encapsulated in individual NAL units and coded independently, each regular slice should be smaller than the MTU in the MTU scheme to avoid splitting the slices into multiple packets. Thus, the goals of parallelization and MTU size matching may impose conflicting requirements on the slice layout within a picture.

[0033] Dependent slices are similar to normal slices, but have shortened slice headers that allow for partitioning of picture treeblock boundaries without breaking intra-picture prediction. Dependent slices therefore allow normal slices to be fragmented into multiple NAL units, which provides reduced end-to-end delay by allowing parts of a normal slice to be sent before the encoding of the entire normal slice is complete.

[0034] A tile is a partitioned portion of an image created by horizontal and vertical boundaries that create tile columns and rows. Tiles may be coded in raster scan order (right to left and top to bottom). The scan order of CTBs is local within a tile. Thus, the CTB in the first tile is coded in raster scan order before proceeding to the CTB in the next tile. Like regular slices, tiles break intra-picture prediction dependencies and entropy decoding dependencies. However, tiles may not be included in individual NAL units, and therefore tiles may not be used for MTU size matching. Each tile may be processed by one processor / core, and inter-processor / inter-core communication used for intra-picture prediction between processing units decoding adjacent tiles may be limited to conveying a shared slice header (when adjacent tiles are in the same slice) and performing loop filtering-related sharing of reconstructed samples and metadata. When more than one tile is included in a slice, the entry point byte offset for each tile other than the first entry point offset within the slice may be signaled in the slice header. For each slice and tile, at least one of the following conditions should be satisfied: 1) all coded treeblocks in a slice belong to the same tile, and 2) all coded treeblocks in a tile belong to the same slice.

[0035] In WPP, an image is partitioned into a single row of CTBs. The entropy decoding and prediction mechanism may use data from CTBs in other rows. Parallel processing is enabled through parallel decoding of CTB rows. For example, the current row may be decoded in parallel with the previous row. However, the decoding of the current row is delayed from the decoding process of the previous row by two CTBs. This delay ensures that data related to the CTB above and to the right of the current CTB in the current row is available before the current CTB is coded. This approach appears as a wavefront when represented graphically. This staggered start allows parallelization across up to the same number of processors / cores as the image contains CTB rows. Because intra-picture prediction between adjacent treeblock rows within a picture is allowed, inter-processor / inter-core communication to enable intra-picture prediction can be substantial. WPP partitioning takes NAL unit size into account. Therefore, WPP does not support MTU size matching. However, to achieve the desired MTU size matching, slicing is typically used with WPP, with some coding overhead.

[0036] A tile may also include a motion constrained tile set. A motion constrained tile set (MCTS) is a tile set designed such that associated motion vectors are constrained to all sample positions within the MCTS and fractional sample positions that require only all sample positions within the MCTS for interpolation. Furthermore, the use of motion vector candidates for temporal motion vector prediction derived from blocks outside the MCTS is prohibited. In this way, each MCTS may be decoded independently without the presence of tiles not included in the MCTS. A temporal MCTS supplemental enhancement information (SEI) message may indicate the presence of an MCTS in a bitstream and be used to signal the MCTS. The MCTS SEI message provides supplemental information that can be used in MCTS sub-bitstream extraction (specified as part of the semantics of the SEI message) to generate a conforming bitstream for the MCTS set. The information includes multiple extraction information sets, each defining multiple MCTS sets, and raw bytes sequence payload (RBSP) bytes of replacement video parameter sets (VPS), sequence parameter sets (SPS), and picture parameter sets (PPS) to be used during the MCTS sub-bitstream extraction process. When extracting a sub-bitstream according to the MCTS sub-bitstream extraction process, the parameter sets (VPS, SPS, and PPS) may be rewritten or replaced, and the slice header may be updated since one or all of the slice address-related syntax elements (including first_slice_segment_in_pic_flag and slice_segment_address) may use different values ​​in the extracted sub-bitstream.

[0037] A picture may also be partitioned into one or more sub-pictures. Partitioning a picture into sub-pictures may allow different portions of the picture to be treated differently from a coding perspective. For example, a sub-picture can be extracted and displayed without extracting other sub-pictures. As other examples, different sub-pictures can be displayed at different resolutions, repositioned relative to each other (e.g., in a videoconferencing application), or coded as separate pictures even though the sub-pictures together contain data from a common picture.

[0038] An exemplary implementation of sub-pictures is as follows: A picture can be partitioned into one or more sub-pictures. A sub-picture is a rectangular or square set of slices / tile groups starting with the slice / tile group with an address equal to 0. Each sub-picture may reference a different PPS, and therefore each sub-picture may use a different partitioning mechanism. Sub-pictures may be treated like pictures in the decoding process. The current reference picture used to decode the current sub-picture may be generated by extracting a region co-located with the current sub-picture from a reference picture in the decoded picture buffer. The extracted region may be a decoded sub-picture, and therefore inter-prediction may be performed between sub-pictures of the same size and location within a picture. A tile group may be a sequence of tiles in the tile raster scan of a sub-picture. The following may be derived to determine the location of a sub-picture within a picture: Each sub-picture may be contained in the next unoccupied position in the CTU raster scan order within a picture large enough to fit the sub-picture within the picture boundary.

[0039] The sub-picture approach used by various video coding systems includes various problems that reduce coding efficiency and / or functionality. This disclosure includes various solutions to such problems. In a first exemplary problem, inter prediction may be performed according to one of several inter prediction modes. Certain inter prediction modes generate candidate lists of motion vector predictors at both the encoder and decoder. This allows the encoder to signal a motion vector by signaling an index from the candidate list instead of signaling the entire motion vector. Furthermore, some systems encode sub-pictures for independent extraction. This allows the current sub-picture to be decoded and displayed without decoding information from other sub-pictures. This can result in errors when a motion vector that points outside the sub-picture is used. This is because the data pointed to by the motion vector may not be decoded and therefore may not be available.

[0040] Therefore, in a first example, a flag indicating that a sub-picture should be treated as a picture is disclosed herein. This flag is set to support separate extraction of sub-pictures. When the flag is set, motion vector predictors obtained from co-located blocks include only motion vectors pointing within the sub-picture. Any motion vector predictors pointing outside the sub-picture are excluded. This ensures that motion vectors pointing outside the sub-picture are not selected and related errors are avoided. Co-located blocks are blocks from pictures other than the current picture. Motion vector predictors from blocks within the current picture (non-co-located blocks) may point outside the sub-picture because other processes, such as interpolation filters, can prevent errors for such motion vector predictors. Therefore, this example provides additional functionality to video encoders / decoders (codecs) by preventing errors when performing sub-picture extraction.

[0041] In a second example, a flag indicating that a subpicture should be treated as a picture is disclosed. When a current subpicture is treated like a picture, the current subpicture should be extracted without reference to other subpictures. Specifically, this example uses a clipping function that is applied when applying an interpolation filter. This clipping function ensures that the interpolation filter does not rely on data from adjacent subpictures to maintain separation between subpictures to support separate extraction. Thus, the clipping function is applied when the flag is set and the motion vector points outside the current subpicture. The interpolation filter is then applied to the result of the clipping function. Thus, this example provides additional functionality to video codecs by preventing errors when performing subpicture extraction. Thus, the first and second examples address the first exemplary problem.

[0042] In a second example problem, a video coding system partitions a picture into subpictures, slices, tiles, and / or coding tree units, which are then partitioned into blocks. Such blocks are then encoded for transmission to a decoder. Decoding such blocks can result in a decoded image containing various types of noise. To correct such problems, the video coding system may apply various filters across block boundaries. These filters can remove blocking, quantization noise, and other undesirable coding artifacts. As noted above, some systems encode subpictures for independent extraction. This allows a current subpicture to be decoded and displayed without decoding information from other subpictures. In such systems, subpictures may be partitioned into blocks for encoding. Thus, block boundaries along subpicture edges may align with subpicture boundaries. In some cases, block boundaries may also align with tile boundaries. Filters may be applied across such block boundaries, and therefore across subpicture and / or tile boundaries. This can cause errors when the current subpicture is extracted independently, as the filtering process may behave in an unexpected way when data from adjacent subpictures is unavailable.

[0043] In a third example, a flag is disclosed herein that controls filtering at the subpicture level. When the flag is set for a subpicture, the filter can be applied across subpicture boundaries. When the flag is not set, the filter is not applied across subpicture boundaries. In this manner, the filter can be turned off for subpictures that are coded for separate extraction, or turned on for subpictures that are coded for display as a group. Thus, this example provides additional functionality to video codecs by preventing filter-related errors when performing subpicture extraction.

[0044] In a fourth example, a flag is disclosed herein that can be set to control filtering at the tile level. When the flag is set for a tile, the filter can be applied across tile boundaries. When the flag is not set, the filter is not applied across tile boundaries. In this manner, the filter can be turned off or on for use at tile boundaries (e.g., while continuing to filter the interior of the tile). Thus, this example provides additional functionality to the video codec by supporting selective filtering across tile boundaries. Thus, the third and fourth examples address the second exemplary problem.

[0045] In a third example problem, a video coding system may partition a picture into sub-pictures. This allows different sub-pictures to be treated differently when coding video. For example, sub-pictures can be extracted and displayed separately, resized independently based on application-level changes, etc. In some cases, sub-pictures may be created by partitioning a picture into tiles and assigning the tiles to sub-pictures. Some video coding systems describe sub-picture boundaries in terms of tiles included in the sub-picture. However, a tiling scheme may not be used in some pictures. Thus, such boundary descriptions may limit the use of sub-pictures to pictures that use tiles.

[0046] In a fifth example, a mechanism for signaling subpicture boundaries in terms of CTBs and / or CTUs is disclosed herein. Specifically, the width and height of a subpicture can be signaled in units of CTBs. The location of the subpicture's top-left CTU can also be signaled as an offset from the picture's top-left CTU, measured in CTBs. The CTU and CTB sizes may be set to predetermined values. Signaling subpicture dimensions and locations in terms of CTBs and CTUs thus provides sufficient information for a decoder to position the subpicture for display. This allows the subpicture to be used even when tiles are not used. This signaling mechanism also avoids complexity and can be coded using relatively few bits. This example therefore provides additional functionality to video codecs by allowing subpictures to be used independently of tiles. Furthermore, this example increases coding efficiency, thus reducing the use of processor, memory, and / or network resources in an encoder and / or decoder. Thus, the fifth example addresses the third exemplary problem.

[0047] In a fourth exemplary problem, a picture can be partitioned into multiple slices for encoding. In some video coding systems, slices are addressed based on their position relative to the picture. Still other video coding systems use the concept of sub-pictures. As noted above, sub-pictures can be treated differently from other sub-pictures from a coding perspective. For example, a sub-picture can be extracted and displayed independently of other sub-pictures. In such cases, slice addresses generated based on picture position may not work properly because a significant number of potential slice addresses are omitted. Some video coding systems address this problem by dynamically rewriting slice headers upon request to change slice addresses to support sub-picture extraction. Such a process can be resource intensive because it may occur each time a user requests to view a sub-picture.

[0048] In a sixth example, slices addressed relative to a subpicture containing the slice are disclosed herein. For example, a slice header may include a subpicture identifier (ID) and the address of each slice included in the subpicture. Furthermore, a sequence parameter set (SPS) may include dimensions of the subpicture, which may be referenced by the subpicture ID. Thus, when separate extraction of a subpicture is required, the slice header does not need to be rewritten. The slice header and SPS contain sufficient information to support positioning of slices within a subpicture for display. This example therefore increases coding efficiency and / or avoids redundant rewriting of slice headers, thus reducing the use of processor, memory, and / or network resources in an encoder and / or decoder. Thus, the sixth example addresses the fourth exemplary problem.

[0049] 1 is a flowchart of an exemplary operational method 100 for coding a video signal. Specifically, a video signal is encoded in an encoder. The encoding process compresses the video signal by using various mechanisms to reduce the video file size. The smaller file size allows the compressed video file to be transmitted to a user while reducing the associated bandwidth overhead. A decoder then decodes the compressed video file to recreate the original video signal for display to the end user. The decoding process generally mirrors the encoding process to allow the decoder to consistently recreate the video signal.

[0050] In step 101, a video signal is input to an encoder. For example, the video signal may be an uncompressed video file stored in memory. As another example, the video file may be captured by a video capture device such as a video camera and encoded to support live streaming of the video. The video file may include both an audio component and a video component. The video component includes a series of image frames that, when viewed in sequence, create the visual impression of movement. A frame includes pixels represented in terms of light, referred to herein as luma components (or luma samples), and colors, referred to herein as chroma components (or color samples). In some examples, the frame may also include depth values ​​to support three-dimensional displays.

[0051] In step 103, the video is partitioned into blocks. Partitioning involves subdividing pixels within each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H Part 2), a frame can first be divided into coding tree units (CTUs), which are blocks of a predetermined size (e.g., 64 pixels by 64 pixels). CTUs contain both luma samples and chroma samples. The coding tree may be used to divide the CTUs into blocks and then recursively subdivide the blocks until a structure that supports further encoding is achieved. For example, the luma component of a frame may be subdivided until each block contains relatively uniform illumination values. Furthermore, the chroma component of a frame may be subdivided until each block contains relatively uniform color values. Thus, the partitioning mechanism varies depending on the content of the video frame.

[0052] In step 105, various compression mechanisms are used to compress the image blocks partitioned in step 103. For example, inter-prediction and / or intra-prediction may be used. Inter-prediction is designed to take advantage of the fact that objects in a common scene tend to appear in consecutive frames. Thus, a block representing an object in a reference frame need not be repeatedly described in adjacent frames. Specifically, an object such as a table may remain in a constant position across multiple frames. Thus, the table may be described once, and adjacent frames can reference back to the reference frame. A pattern matching mechanism may be used to match objects across multiple frames. Furthermore, a moving object may be represented across multiple frames, for example, due to object motion or camera motion. As a specific example, a video may show a car moving across the screen across multiple frames. To describe such motion, a motion vector may be used. A motion vector is a two-dimensional vector that provides an offset from the coordinates of the object in the frame to the coordinates of the object in the reference frame. Thus, inter-prediction can encode an image block in a current frame as a set of motion vectors that indicate an offset from a corresponding block in a reference frame.

[0053] Intra prediction encodes blocks within a common frame. It takes advantage of the fact that luma and chroma components tend to be concentrated within a frame. For example, green fragments in a section of a tree tend to be located adjacent to similar green fragments. Intra prediction uses multiple directional prediction modes (e.g., 33 in HEVC), planar mode and direct current (DC) mode. Directional mode indicates that the current block is similar / identical to samples of neighboring blocks in the corresponding direction. Planar mode indicates that a series of blocks (e.g., a plane) along a row / column can be interpolated based on neighboring blocks at the end of the row. Planar mode effectively indicates a smooth transition of light / color across a row / column by using a relatively constant slope in changing values. DC mode is used for boundary smoothing and indicates that the block is similar / identical to the average value associated with samples of all neighboring blocks associated with the angular direction of the directional prediction mode. Therefore, intra prediction blocks can represent image blocks as various related prediction mode values ​​instead of their actual values. Furthermore, inter-predicted blocks can represent image blocks as motion vector values ​​instead of actual values. In either case, the predicted block may not accurately represent the image block in some cases. Any differences are stored in a residual block. To further compress the file, a transform may be applied to the residual block.

[0054] Various filtering techniques may be applied in step 107. In HEVC, filters are applied according to an in-loop filtering scheme. The block-based prediction described above may result in the generation of blocky images at the decoder. Furthermore, the block-based prediction scheme may encode a block and then reconstruct the encoded block for later use as a reference block. The in-loop filtering scheme iteratively applies noise suppression filters, deblocking filters, adaptive loop filters, and sample adaptive offset (SAO) filters to a block / frame. These filters mitigate such blocking artifacts so that the encoded file can be accurately reproduced. Furthermore, these filters mitigate artifacts in the reconstructed reference block so that the artifacts are less likely to create further artifacts in subsequent blocks that are coded based on the reconstructed reference block.

[0055] Once the video signal has been segmented, compressed, and filtered, the resulting data is encoded into a bitstream in step 109. The bitstream includes the above data and any signaling data desired to support proper video signal reproduction at a decoder. For example, such data may include segmentation data, prediction data, residual blocks, and various flags that provide coding instructions to the decoder. The bitstream may be stored in memory for transmission to a decoder upon request. The bitstream may also be broadcast and / or multicast to multiple decoders. Creation of the bitstream is an iterative process. Thus, steps 101, 103, 105, 107, and 109 may occur sequentially and / or simultaneously across many frames and blocks. The order shown in FIG. 1 is presented for clarity and ease of explanation and is not intended to limit the video coding process to any particular order.

[0056] The decoder receives the bitstream and begins the decoding process in step 111. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into corresponding syntax and video data. In step 111, the decoder uses syntax data from the bitstream to determine the frame partition. The partition should match the result of the block partitioning in step 103. We now describe the entropy encoding / decoding used in step 111. The encoder makes many choices during the compression process, such as selecting a block partitioning scheme from several possible options based on the spatial location of values ​​within the input image. Signaling the exact choice may use multiple bins. As used herein, a bin is a binary value (e.g., a bit value that can change depending on the context) that is treated as a variable. Entropy coding allows the encoder to discard any options that are clearly infeasible in a particular case, leaving a set of allowable options. Each allowable option is then assigned a codeword. The length of the codeword is based on the number of allowable options (e.g., one bin for two options, two bins for three to four options, etc.). The encoder then encodes a codeword for the selected option. This scheme reduces the size of the codeword, as it is desirable for the codeword to be large enough to uniquely indicate a selection from a small subset of allowable options, as opposed to uniquely indicating a selection from a potentially large set of all possible options. The decoder then decodes the selection by determining the set of allowable options in a similar manner to the encoder. By determining the set of allowable options, the decoder can read the codeword and determine the selection made by the encoder.

[0057] In step 113, the decoder performs block decoding. Specifically, the decoder uses an inverse transform to generate a residual block. The decoder then uses the residual block and a corresponding prediction block to reconstruct an image block according to the partition. The prediction block may include both an intra-prediction block and an inter-prediction block as generated by the encoder in step 105. The reconstructed image block is then positioned in a frame of the reconstructed video signal according to the partition data determined in step 111. The syntax for step 113 may also be signaled in the bitstream via entropy coding, as described above.

[0058] In step 115, filtering is performed on the frames of the reconstructed video signal in a manner similar to step 107 in the encoder. For example, noise suppression filters, deblocking filters, adaptive loop filters, and SAO filters may be applied to the frames to remove blocking artifacts. Once the frames have been filtered, the video signal can be output to a display in step 117 for viewing by an end user.

[0059] 2 is a schematic diagram of an exemplary coding and decoding (codec) system 200 for video coding. Specifically, codec system 200 provides functionality to support the implementation of operational method 100. Codec system 200 is generalized to show components used in both encoders and decoders. Codec system 200 receives and segments a video signal, as described with reference to steps 101 and 103 in operational method 100, which results in a segmented video signal 201. When functioning as an encoder, codec system 200 then compresses the segmented video signal 201 into a coded bitstream, as described with reference to steps 105, 107, and 109 in method 100. When functioning as a decoder, codec system 200 generates an output video signal from the bitstream, as described with reference to steps 111, 113, 115, and 117 in operational method 100. Codec system 200 includes an overall coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, an intra-picture prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control analysis component 227, an in-loop filter component 225, a decoded picture buffer component 223, and a header format and context adaptive binary arithmetic coding (CABAC) component 231. These components are coupled as shown. In FIG. 2, black lines indicate the movement of data to be encoded / decoded, and dashed lines indicate the movement of control data that controls the operation of other components. The components of codec system 200 may all be present in an encoder. A decoder may include a subset of the components of codec system 200.For example, the decoder may include an intra-picture prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoded picture buffer component 223. These components are now described.

[0060] The partitioned video signal 201 is a captured video sequence that has been partitioned into blocks of pixels by a coding tree. The coding tree uses various partitioning modes to subdivide the blocks of pixels into smaller blocks of pixels. These blocks can then be further subdivided into smaller blocks. The blocks may be referred to as nodes on the coding tree. Larger parent nodes are divided into smaller child nodes. The number of times a node is subdivided is referred to as the depth of the node / coding tree. In some cases, the partitioned blocks can be included in a coding unit (CU). For example, a CU can be a subpart of a CTU that includes a luma block, a red-difference chroma (Cr) block, and a blue-difference chroma (Cb) block, along with corresponding syntax instructions for the CU. Partitioning modes may include a binary tree (BT), a triple tree (TT), and a quad tree (QT), which are used to partition a node into two, three, or four child nodes of varying shapes, respectively, depending on the partitioning mode used. The segmented video signal 201 is forwarded to an overall coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, a filter control analysis component 227, and a motion estimation component 221 for compression.

[0061] The overall coder control component 211 is configured to make decisions related to the coding of images of a video sequence into a bitstream according to application constraints. For example, the overall coder control component 211 manages the optimization of bitrate / bitstream size versus reproduction quality. Such decisions may be made based on storage space / bandwidth availability and image resolution requirements. The overall coder control component 211 also manages buffer utilization, taking transmission rate into account, to mitigate buffer underrun and overrun issues. To manage these issues, the overall coder control component 211 manages segmentation, prediction, and filtering by other components. For example, the overall coder control component 211 may dynamically increase compression complexity to increase resolution and bandwidth usage, or decrease compression complexity to decrease resolution and bandwidth usage. Thus, the overall coder control component 211 controls other components of the codec system 200 to balance bitrate concerns with video signal reproduction quality. The overall coder control component 211 generates control data that controls the operation of other components. Control data is also forwarded to the header format and CABAC component 231 to be encoded into the bitstream for signaling parameters for decoding at the decoder.

[0062] The partitioned video signal 201 is also sent to a motion estimation component 221 and a motion compensation component 219 for inter-prediction. A frame or slice of the partitioned video signal 201 may be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform inter-predictive coding of the received video blocks relative to one or more blocks in one or more reference frames to provide temporal prediction. The codec system 200 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0063] The motion estimation component 221 and the motion compensation component 219 may be highly integrated but are illustrated separately for conceptual purposes. Motion estimation, performed by the motion estimation component 221, is the process of generating motion vectors that estimate motion for video blocks. A motion vector may indicate, for example, the displacement of a coded object relative to a predictive block. A predictive block is a block found to closely match a block to be coded in terms of pixel differences. A predictive block may also be referred to as a reference block. Such pixel differences may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metrics. HEVC uses several coded objects, including CTUs, coding tree blocks (CTBs), and CUs. For example, a CTU can be divided into CTBs, which can then be divided into CBs for inclusion in CUs. A CU can be coded as a prediction unit (PU) containing prediction data and / or a transform unit (TU) containing transform residual data for the CU. The motion estimation component 221 generates motion vectors, PUs, and TUs by using rate-distortion analysis as part of a rate-distortion optimization process. For example, the motion estimation component 221 may determine multiple reference blocks, multiple motion vectors, etc. for the current block / frame and select the reference block, motion vector, etc. with the best rate-distortion characteristics. The best rate-distortion characteristics balance both the quality of the video reproduction (e.g., the amount of data loss due to compression) and the coding efficiency (e.g., the size of the final encoding).

[0064] In some examples, the codec system 200 may calculate values ​​for sub-integer pixel positions of reference pictures stored in the decoded picture buffer component 223. For example, the video codec system 200 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference pictures. Accordingly, the motion estimation component 221 may perform motion search for whole-pixel and fractional pixel positions and output motion vectors with fractional pixel accuracy. The motion estimation component 221 calculates motion vectors for PUs of video blocks in inter-coded slices by comparing the positions of the PUs with the positions of predictive blocks in the reference pictures. The motion estimation component 221 outputs the calculated motion vectors as motion data to the header format and CABAC component 231 and motion to the motion compensation component 219 for encoding.

[0065] The motion compensation performed by motion compensation component 219 may involve fetching or generating a predictive block based on a motion vector determined by motion estimation component 221. Similarly, in some examples, motion estimation component 221 and motion compensation component 219 may be functionally integrated. Upon receiving a motion vector for a PU of a current video block, motion compensation component 219 may identify a predictive block to which the motion vector points. A residual video block is then formed by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being coded to form pixel difference values. Typically, motion estimation component 221 performs motion estimation on the luma component, and motion compensation component 219 uses the motion vector calculated based on the luma component for both the chroma and luma components. The predictive block and residual block are forwarded to transform scaling and quantization component 213.

[0066] The partitioned video signal 201 is also sent to an intra-picture estimation component 215 and an intra-picture prediction component 217. Like the motion estimation component 221 and the motion compensation component 219, the intra-picture estimation component 215 and the intra-picture prediction component 217 may be highly integrated but are illustrated separately for conceptual purposes. The intra-picture estimation component 215 and the intra-picture prediction component 217 intra-predict the current block relative to blocks within the current frame, as an alternative to the inter-prediction performed by the motion estimation component 221 and the motion compensation component 219 between frames, as described above. In particular, the intra-picture estimation component 215 determines the intra-prediction mode to use to encode the current block. In some examples, the intra-picture estimation component 215 selects an appropriate intra-prediction mode for encoding the current block from multiple tested intra-prediction modes. The selected intra-prediction mode is then forwarded to the header format and CABAC component 231 for encoding.

[0067] For example, the intra picture estimation component 215 may calculate rate-distortion values ​​for various tested intra prediction modes using rate-distortion analysis and select the intra prediction mode with the best rate-distortion characteristics among the tested modes. The rate-distortion analysis generally determines the amount of distortion (or error) between the original uncoded block coded to generate the coded block and the coded block, and the bit rate (e.g., number of bits) used to generate the coded block. The intra picture estimation component 215 may calculate a ratio from the distortion and rate for the various coded blocks to determine which intra prediction mode exhibits the best rate-distortion value for the block. Furthermore, the intra picture estimation component 215 may be configured to code the depth blocks of the depth map using a depth modeling mode (DMM) based on rate-distortion optimization (RDO).

[0068] The intra-picture prediction component 217 may generate a residual block from the prediction block based on the selected intra-picture prediction mode determined by the intra-picture estimation component 215 when implemented in an encoder, or may read the residual block from the bitstream when implemented in a decoder. The residual block contains the value differences between the prediction block and the original block, represented as a matrix. The residual block is then forwarded to the transform scaling and quantization component 213. The intra-picture estimation component 215 and the intra-picture prediction component 217 may operate on both the luma and chroma components.

[0069] The transform scaling and quantization component 213 is configured to further compress the residual block. The transform scaling and quantization component 213 applies a transform, such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform, to the residual block to generate a video block containing residual transform coefficient values. A wavelet transform, an integer transform, a subband transform, or other types of transforms may also be used. The transform may convert the residual information from the pixel value domain to a transform domain, such as the frequency domain. The transform scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling involves applying a scale factor to the residual information so that different frequency information is quantized with different granularity, which may affect the final visual quality of the reproduced video. The transform scaling and quantization component 213 is also configured to quantize the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the transform scaling and quantization component 213 may then perform a scan of a matrix containing the quantized transform coefficients. The quantized transform coefficients are forwarded to the header format and CABAC component 231 for encoding into a bitstream.

[0070] The scaling and inverse transform component 229 applies the inverse operations of the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 applies, for example, inverse scaling, transform, and / or quantization to reconstruct a residual block in the pixel domain for later use as a reference block that can become a prediction block for another current block. The motion estimation component 221 and / or motion compensation component 219 may calculate a reference block by adding the residual block back to the corresponding prediction block for use in motion estimation of a later block / frame. A filter is applied to the reconstructed reference block to mitigate artifacts created during scaling, quantization, and transform. Otherwise, such artifacts could cause inaccurate predictions (and create further artifacts) when subsequent blocks are predicted.

[0071] The filter control analysis component 227 and the in-loop filter component 225 apply filters to residual blocks and / or reconstructed image blocks. For example, a transformed residual block from the scaling and inverse transform component 229 may be combined with a corresponding prediction block from the intra-picture prediction component 217 and / or the motion compensation component 219 to reconstruct the original image block. The filter may then be applied to the reconstructed image block. In some examples, the filter may be applied to the residual block instead. Like the other components in FIG. 2, the filter control analysis component 227 and the in-loop filter component 225 are highly integrated and may be implemented together, but are shown separately for conceptual purposes. The filters applied to reconstructed reference blocks are applied to specific spatial regions and include multiple parameters to adjust how such filters are applied. The filter control analysis component 227 analyzes the reconstructed reference blocks to determine where such filters should be applied and sets the corresponding parameters. Such data is forwarded to the header format and CABAC component 231 as filter control data for encoding. The in-loop filter component 225 applies such filters based on the filter control data. The filters may include deblocking filters, noise suppression filters, SAO filters, and adaptive loop filters. Such filters may be applied in the spatial / pixel domain (e.g., reconstructed pixel blocks) or the frequency domain, depending on the example.

[0072] When operating as an encoder, the reconstructed image blocks, residual blocks, and / or prediction blocks after filtering are stored in the decoded picture buffer component 223 for later use in motion estimation, as described above. When operating as a decoder, the decoded picture buffer component 223 stores the reconstructed and filtered blocks and forwards them to the display as part of the output video signal. The decoded picture buffer component 223 may be any memory device capable of storing prediction blocks, residual blocks, and / or reconstructed image blocks.

[0073] The header format and CABAC component 231 receives data from various components of the codec system 200 and encodes such data into a coded bitstream for transmission to a decoder. Specifically, the header format and CABAC component 231 generates various headers for encoding control data, such as global control data and filter control data. Additionally, prediction data, including intra-prediction and motion data, and residual data in the form of quantized transform coefficient data are all encoded into the bitstream. The final bitstream contains all information required by a decoder to reconstruct the original partitioned video signal 201. Such information may also include an intra-prediction mode index table (also called a codeword mapping table), definitions of coding contexts for various blocks, indications of the most probable intra-prediction mode, indications of partition information, etc. Such data may be encoded using entropy coding. For example, the information may be encoded using context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. Following entropy coding, the coded bitstream may be transmitted to another device (e.g., a video decoder) or archived for later transmission or retrieval.

[0074] 3 is a block diagram illustrating an exemplary video encoder 300. Video encoder 300 may be used to implement the encoding functionality of codec system 200 and / or to implement steps 101, 103, 105, 107, and / or 109 of method of operation 100. Encoder 300 segments an input video signal, resulting in a segmented video signal 301, which is substantially similar to segmented video signal 201. Segmented video signal 301 is then compressed and encoded into a bitstream by components of encoder 300.

[0075] Specifically, the partitioned video signal 301 is forwarded to an intra-picture prediction component 317 for intra prediction. The intra-picture prediction component 317 may be substantially similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. The partitioned video signal 301 is also forwarded to a motion compensation component 321 for inter prediction based on reference blocks in a decoded picture buffer component 323. The motion compensation component 321 may be substantially similar to the motion estimation component 221 and the motion compensation component 219. The prediction blocks and residual blocks from the intra-picture prediction component 317 and the motion compensation component 321 are forwarded to a transform and quantization component 313 for transforming and quantizing the residual blocks. The transform and quantization component 313 may be substantially similar to the transform scaling and quantization component 213. The transformed and quantized residual blocks and corresponding prediction blocks (along with associated control data) are forwarded to an entropy coding component 331 for coding into a bitstream. The entropy coding component 331 may have a header format substantially similar to the CABAC component 231 .

[0076] The transformed and quantized residual block and / or the corresponding prediction block are also transferred from the transform and quantization component 313 to the inverse transform and quantization component 329 to reconstruct them into reference blocks for use by the motion compensation component 321. The inverse transform and quantization component 329 may be substantially similar to the scaling and inverse transform component 229. An in-loop filter in the in-loop filter component 325 is also applied to the residual block and / or the reconstructed reference block, depending on the example. The in-loop filter component 325 may be substantially similar to the filter control analysis component 227 and the in-loop filter component 225. The in-loop filter component 325 may include multiple filters, as described with respect to the in-loop filter component 225. The filtered block is then stored in the decoded picture buffer component 323 for use as a reference block by the motion compensation component 321. The decoded picture buffer component 323 may be substantially similar to the decoded picture buffer component 223.

[0077] 4 is a block diagram illustrating an exemplary video decoder 400. Video decoder 400 may be used to implement the decoding functionality of codec system 200 and / or to implement steps 111, 113, 115, and / or 117 of method of operation 100. Decoder 400 receives a bitstream from, for example, encoder 300 and generates a reconstructed output video signal based on the bitstream for display to an end user.

[0078] The bitstream is received by the entropy decoding component 433. The entropy decoding component 433 is configured to implement an entropy decoding scheme, such as CAVLC, CABAC, SBAC, PIPE coding, or other entropy coding techniques. For example, the entropy decoding component 433 may use header information to provide context for interpreting additional data encoded as codewords in the bitstream. The decoding information may include any desired information for decoding the video signal, such as global control data, filter control data, partition information, motion data, prediction data, and quantized transform coefficients from the residual block. The quantized transform coefficients are forwarded to the inverse transform and quantization component 429 for reconstruction into the residual block. The inverse transform and quantization component 429 may be similar to the inverse transform and quantization component 329.

[0079] The reconstructed residual blocks and / or predictive blocks are forwarded to an intra-picture prediction component 417 for reconstruction into image blocks based on intra-prediction operations. The intra-picture prediction component 417 may be similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. Specifically, the intra-picture prediction component 417 uses a prediction mode to identify reference blocks within a frame and applies the residual blocks to the result to reconstruct intra-predicted image blocks. The reconstructed intra-predicted image blocks and / or residual blocks and corresponding inter-prediction data are forwarded to a decoded picture buffer component 423 via an in-loop filter component 425, which may be substantially similar to the decoded picture buffer component 223 and the in-loop filter component 225, respectively. The in-loop filter component 425 filters the reconstructed image blocks, residual blocks, and / or predictive blocks, and such information is stored in the decoded picture buffer component 423. The reconstructed image blocks from the decoded picture buffer component 423 are forwarded to the motion compensation component 421 for inter-prediction. The motion compensation component 421 may be substantially similar to the motion estimation component 221 and / or the motion compensation component 219. Specifically, the motion compensation component 421 uses motion vectors from a reference block to generate a prediction block and applies a residual block to the result to reconstruct the image block. The resulting reconstructed blocks may also be forwarded to the decoded picture buffer component 423 via an in-loop filter component 425. The decoded picture buffer component 423 continues to store further reconstructed image blocks, which can be reconstructed into frames via partition information. Such frames may also be arranged in a sequence. The sequence is output to a display as a reconstructed output video signal.

[0080] 5A is a schematic diagram illustrating an example picture 500 partitioned into sub-pictures 510. For example, picture 500 can be partitioned for encoding by codec system 200 and / or encoder 300, and can be partitioned for decoding by codec system 200 and / or decoder 400. As another example, picture 500 may be partitioned by the encoder in step 103 of method 100 for use by the decoder in step 111.

[0081] Picture 500 is an image that shows a complete visual portion of a video sequence at a specified time position. Picture 500 may also be referred to as an image and / or a frame. Picture 500 may be designated by a picture order count (POC). POC is an index that indicates the output / display order of picture 500 in a video sequence. Picture 500 may be partitioned into sub-pictures 510. Sub-pictures 510 are rectangular or square regions of one or more slice / tile groups within picture 500. Sub-pictures 510 are optional; therefore, some video sequences include sub-pictures 510 while others do not. Although four sub-pictures 510 are shown, picture 500 may be partitioned into any number of sub-pictures 510. The partitioning of sub-pictures 510 may be consistent throughout the entire coded video sequence.

[0082] Subpictures 510 may be used to allow different regions of picture 500 to be treated differently. For example, a given subpicture 510 may be independently extracted and transmitted to a decoder. As a specific example, a user using a virtual reality (VR) headset may view a subset of picture 500, which may provide the user with the impression of being physically present in the space shown in picture 500. In such cases, streaming only the subpictures 510 that are likely to be displayed to the user may increase coding efficiency. As another example, different subpictures 510 may be treated differently in certain applications. As a specific example, a videoconferencing application may display an active speaker at a more prominent location and higher resolution than users who are not currently speaking. Positioning different users within different subpictures 510 supports real-time reconstruction of the display image to support this functionality.

[0083] Each subpicture 510 can be identified by a unique subpicture ID, which may be consistent across the entire CVS. For example, the top-left subpicture 510 of picture 500 may have a subpicture ID of 0. In such a case, the top-left subpicture 510 of any picture 500 in the sequence can be referenced by a subpicture ID of 0. Furthermore, each subpicture 510 may have a defined configuration, which may be consistent across the entire CVS. For example, a subpicture 510 may include a height, width, and / or offset. The height and width describe the size of the subpicture 510, and the offset describes the position of the subpicture 510. For example, the sum of all the widths of the subpictures 510 in a row is the width of the picture 500. Furthermore, the sum of all the heights of the subpictures 510 in a column is the height of the picture 500. Furthermore, the offset indicates the position of the top-left corner of the subpicture 510 relative to the top-left corner of the picture 500. The height, width, and offset of a subpicture 510 provide sufficient information to position the corresponding subpicture 510 within the picture 500. Since the partitioning of a subpicture 510 may be consistent across the entire CVS, parameters related to the subpicture may be included in a sequence parameter set (SPS).

[0084] FIG. 5B is a schematic diagram illustrating an exemplary subpicture 510 partitioned into slices 515. As shown, the subpicture 510 of the picture 500 may include one or more slices 515. A slice 515 is an integer number of complete tiles or an integer number of contiguous complete CTU rows within the tiles of the picture that are exclusively contained in a single network abstraction layer (NAL) unit. Although four slices 515 are shown, the subpicture 510 may include any number of slices 515. The slices 515 contain visual data specific to the picture 500 of a given POC. Accordingly, parameters associated with the slices 515 may be included in a picture parameter set (PPS) and / or a slice header.

[0085] FIG. 5C is a schematic diagram illustrating an example slice 515 partitioned into tiles 517. As shown, a slice 515 of a picture 500 may include one or more tiles 517. The tiles 517 may be created by partitioning the picture 500 into rectangular rows and columns. Thus, a tile 517 is a rectangular or square region of a CTU within a particular tile column and a particular tile row within a picture. Tiling is optional; thus, some video sequences include tiles 517, while others do not. Although four tiles 517 are shown, a slice 515 may include any number of tiles 517. The tiles 517 may include visual data specific to the slice 515 of the picture 500 of a given POC. In some cases, a slice 515 may also be included in a tile 517. Thus, parameters related to the tile 517 may be included in the PPS and / or slice header.

[0086] 5D is a schematic diagram illustrating an example slice 515 partitioned into CTUs 519. As shown, a slice 515 (or tile 517 of a slice 515) of a picture 500 may include one or more CTUs 519. A CTU 519 is a region of a picture 500 subdivided by a coding tree to create coding blocks that are encoded / decoded. A CTU 519 may include luma samples for a monochrome picture 500 or a combination of luma and chroma samples for a color picture 500. A grouping of luma or chroma samples that can be partitioned by a coding tree is called a coding tree block (CTB) 518. Thus, a CTU 519 includes a CTB 518 of luma samples and two corresponding CTBs 518 of chroma samples for a picture 500 having a three-sample arrangement, or a CTB 518 of samples for a monochrome picture or a picture coded using a syntax structure used to code three separate color planes and samples.

[0087] As shown above, picture 500 may be partitioned into sub-pictures 510, slices 515, tiles 517, CTUs 519, and / or CTBs 518, which are then partitioned into blocks. Such blocks are then encoded for transmission to a decoder. Decoding such blocks can result in a decoded image containing various types of noise. To correct such problems, a video coding system may apply various filters across block boundaries. These filters can remove blocking, quantization noise, and other undesirable coding artifacts. As noted above, sub-pictures 510 may be used when performing independent extraction. In this case, the current sub-picture 510 may be decoded and displayed without decoding information from other sub-pictures 510. Thus, block boundaries along sub-picture 510 edges may align with sub-picture boundaries. In some cases, block boundaries may also align with tile boundaries. Filters may be applied across such block boundaries, and thus across sub-picture and / or tile boundaries. This can cause errors when the current subpicture 510 is extracted independently, as the filtering process may behave in an unexpected way when data from adjacent subpictures 510 is unavailable.

[0088] To address these issues, a flag may be used to control filtering at the subpicture 510 level. For example, the flag may be indicated as loop_filter_across_subpic_enabled_flag. When the flag is set for a subpicture 510, the filter can be applied across the corresponding subpicture boundary. When the flag is not set, the filter is not applied across the corresponding subpicture boundary. In this way, the filter can be turned off for subpictures 510 that are coded for separate extraction, or turned on for subpictures 510 that are coded for display as a group. Another flag can be set to control filtering at the tile 517 level. The flag may be indicated as loop_filter_across_tiles_enabled_flag. When the flag is set for a tile 517, the filter can be applied across tile boundaries. When the flag is not set, the filter is not applied across tile boundaries. In this way, the filter can be turned off or on for use at tile boundaries (e.g., while continuing to filter the interior of the tile).

[0089] Also, as noted above, tiling is optional. However, some video coding systems describe subpicture boundaries in terms of tiles 517 contained in the subpicture 510. In such systems, the subpicture boundary description in terms of tiles 517 restricts the use of the subpicture 510 to pictures 500 that use the tiles 517. To expand the applicability of the subpicture 510, the subpicture 510 may be described in terms of boundaries in terms of CTBs 518 and / or CTUs 519. Specifically, the width and height of the subpicture 510 may be signaled in units of CTBs 518. Furthermore, the position of the top-left CTU 519 of the subpicture 510 may be signaled as an offset from the top-left CTU 519 of the picture 500 as measured in CTBs 518. The CTU 519 and CTB 518 sizes may be set to predetermined values. Thus, signaling the subpicture dimensions and position with respect to the CTB 518 and CTU 519 provides sufficient information for the decoder to position the subpicture 510 for display. This allows the subpicture 510 to be used even when the tile 517 is not used. As used herein, a filter is applied across a subpicture 510 or tile 517 boundary when the filter is applied to samples on either side of the boundary.

[0090] Furthermore, some video coding systems address slices 515 based on their position relative to the picture 500. This creates a problem when a subpicture 510 is coded for independent extraction and display. In such cases, the slices 515 and corresponding addresses associated with the omitted subpicture 510 are also omitted. Omitting the address of a slice 515 can prevent a decoder from properly locating the slice 515. Some video coding systems address this problem by dynamically rewriting the address in the slice header associated with the slice 515. Because a user may request any subpicture, such rewriting occurs every time the user requests video, which is extremely resource-intensive. To overcome this problem, a slice 515 is addressed relative to the subpicture 510 that contains it when the subpicture 510 is used. For example, the slice 515 can be identified by an index or other value that is specific to the subpicture 510 that contains it. The slice address can be coded in the slice header associated with the slice 515. The sub-picture ID of the sub-picture 510 that contains the slice 515 can also be coded in the slice header. Furthermore, the dimensions / configuration of the sub-picture 510 can be coded in the SPS along with the sub-picture ID. Thus, the decoder can obtain the sub-picture 510 configuration from the SPS based on the sub-picture ID and position the slice 515 in the sub-picture 510 without referencing the complete picture 500. Therefore, rewriting of the slice header can be omitted when the sub-picture 510 is extracted, which significantly reduces resource usage in the encoder, decoder, and / or corresponding slicer.

[0091] Once the picture 500 is partitioned into CTBs 518 and / or CTUs 519, the CTBs 518 and / or CTUs 519 can be further divided into coding blocks. The coding blocks can then be coded according to intra-prediction and / or inter-prediction. This disclosure also includes improvements related to inter-prediction mechanisms. Inter-prediction can be performed in several different modes, which can operate according to unidirectional inter-prediction and / or bidirectional inter-prediction.

[0092] 6 is a schematic diagram illustrating an example of unidirectional inter prediction 600 that may be performed to determine motion vectors (MVs), for example, in block compression step 105, block decoding step 113, motion estimation component 221, motion compensation component 219, motion compensation component 321, and / or motion compensation component 421. For example, unidirectional inter prediction 600 may be used to determine motion vectors for coded and / or decoded blocks created when partitioning a picture such as picture 500.

[0093] The unidirectional inter prediction 600 uses a reference frame 630 having a reference block 631 to predict a current block 611 in a current frame 610. The reference frame 630 may be temporally positioned after the current frame 610 (e.g., as a subsequent reference frame) as shown, but in some examples, it may be temporally positioned before the current frame 610 (e.g., as a preceding reference frame). The current frame 610 is an exemplary frame / picture being encoded / decoded at a particular time. The current frame 610 includes an object in the current block 611 that matches an object in the reference block 631 of the reference frame 630. The reference frame 630 is a frame used as a reference for encoding the current frame 610, and the reference block 631 is a block in the reference frame 630 that includes an object that is also included in the current block 611 of the current frame 610.

[0094] The current block 611 is any coding unit being coded / decoded at a specified point in the coding process. The current block 611 may be an entire partitioned block or a sub-block when using affine inter prediction mode. The current frame 610 is separated from the reference frame 630 by some temporal distance (TD) 633. The TD 633 indicates the amount of time between the current frame 610 and the reference frame 630 in a video sequence and may be measured in units of frames. Prediction information for the current block 611 may reference the reference frame 630 and / or the reference block 631 by a reference index that indicates the direction and temporal distance between the frames. Over the period represented by the TD 633, an object in the current block 611 moves from a position in the current frame 610 to another position in the reference frame 630 (e.g., the position of the reference block 631). For example, the object may move along a motion trajectory 613, which is the direction of the object's movement over time. The motion vector 635 describes the direction and magnitude of the object's motion along the motion trajectory 613 across the TD 633. Thus, the coded motion vector 635, the reference block 631, and the residual comprising the difference between the current block 611 and the reference block 631 provide sufficient information to reconstruct the current block 611 and position it within the current frame 610.

[0095] 7 is a schematic diagram illustrating an example of bidirectional inter prediction 700 that may be performed to determine MVs, for example, in block compression step 105, block decoding step 113, motion estimation component 221, motion compensation component 219, motion compensation component 321, and / or motion compensation component 421. For example, bidirectional inter prediction 700 may be used to determine motion vectors for coded and / or decoded blocks created when partitioning a picture such as picture 500.

[0096] Bidirectional inter prediction 700 is similar to unidirectional inter prediction 600, but uses a pair of reference frames to predict a current block 711 in a current frame 710. Thus, the current frame 710 and the current block 711 are substantially similar to the current frame 610 and the current block 611, respectively. The current frame 710 is temporally positioned between a previous reference frame 720, which occurs before the current frame 710 in the video sequence, and a subsequent reference frame 730, which occurs after the current frame 710 in the video sequence. The previous reference frame 720 and the subsequent reference frame 730 are otherwise substantially similar to the reference frame 630.

[0097] A current block 711 is matched with a previous reference block 721 in a previous reference frame 720 and a subsequent reference block 731 in a subsequent reference frame 730. Such a match indicates that an object moves from the position of the previous reference block 721 along a motion trajectory 713 through the current block 711 to the position of the subsequent reference block 731 over the course of the video sequence. The current frame 710 is separated from the previous reference frame 720 by some previous temporal distance (TD0) 723 and from the subsequent reference frame 730 by some subsequent temporal distance (TD1) 733. TD0 723 indicates the amount of time, in units of frames, between the previous reference frame 720 and the current frame 710 in the video sequence. TD1 733 indicates the amount of time, in units of frames, between the current frame 710 and the subsequent reference frame 730 in the video sequence. Thus, the object moves along motion trajectory 713 from previous reference block 721 to current block 711 over the period indicated by TD0 723. The object also moves along motion trajectory 713 from current block 711 to subsequent reference block 731 over the period indicated by TD1 733. Prediction information for current block 711 may refer to previous reference frame 720 and / or previous reference block 721 and subsequent reference frame 730 and / or subsequent reference block 731 by a pair of reference indexes indicating the direction and time distance between the frames.

[0098] A previous motion vector (MV0) 725 describes the direction and magnitude of an object's motion along a motion trajectory 713 across TD0 723 (e.g., between a previous reference frame 720 and a current frame 710). A subsequent motion vector (MV1) 735 describes the direction and magnitude of an object's motion along a motion trajectory 713 across TD1 733 (e.g., between a current frame 710 and a subsequent reference frame 730). Thus, in bidirectional inter prediction 700, a current block 711 can be coded and reconstructed by using a previous reference block 721 and / or a subsequent reference block 731, MV0 725, and MV1 735.

[0099] In both merge mode and advanced motion vector prediction (AMVP) mode, a candidate list is generated by adding candidate motion vectors to the candidate list in an order defined by a candidate list determination pattern. Such candidate motion vectors may include motion vectors generated according to unidirectional inter prediction 600, bidirectional inter prediction 700, or a combination thereof. Specifically, motion vectors are generated for neighboring blocks when such blocks are coded. Such motion vectors are added to a candidate list for a current block, and a motion vector for the current block is selected from the candidate list. The motion vector can then be signaled as the index of the selected motion vector in the candidate list. A decoder can build the candidate list using the same process as an encoder and determine the selected motion vector from the candidate list based on the signaled index. Thus, the candidate motion vectors include motion vectors generated according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700, depending on which technique is used when such neighboring blocks are coded.

[0100] 8 is a schematic diagram illustrating an example 800 of coding a current block 801 based on candidate motion vectors from neighboring coded blocks 802. Using the operating method 100 of the encoder 300 and / or decoder 400 and / or functionality of the codec system 200, the neighboring blocks 802 can be used to generate a candidate list. Such a candidate list can be used in inter-prediction by unidirectional inter-prediction 600 and / or bidirectional inter-prediction 700. The candidate list can then be used to encode / decode the current block 801, which may be generated by partitioning a picture such as picture 500.

[0101] The current block 801 is a block being coded by an encoder or decoded by a decoder at a specified time, depending on the example. The coded block 802 is a block that has already been coded at a specified time. Therefore, the coded block 802 is potentially available for use when generating the candidate list. The current block 801 and the coded block 802 may be included in a common frame and / or in temporally adjacent frames. When the coded block 802 is included in a common frame with the current block 801, the coded block 802 has a boundary that is immediately adjacent (e.g., touching) the boundary of the current block 801. When the coded block 802 is included in a temporally adjacent frame, the coded block 802 is located in the same position in the temporally adjacent frame as the current block 801 in the current frame. The candidate list can be generated by adding the motion vector from the coded block 802 as a candidate motion vector. The current block 801 can then be coded by selecting a candidate motion vector from the candidate list and signaling the index of the selected candidate motion vector.

[0102] 9 is a schematic diagram illustrating an example pattern 900 for determining a candidate list of motion vectors. Specifically, the operating method 100 of the encoder 300 and / or decoder 400 and / or use of the functionality of the codec system 200 can use the candidate list determination pattern 900 to use in generating a candidate list 911 for encoding a current block 801 partitioned from a picture 500. The resulting candidate list 911 can be a merge candidate list or an AMVP candidate list, which can be used in inter prediction with unidirectional inter prediction 600 and / or bidirectional inter prediction 700.

[0103] When encoding a current block 901, the candidate list determination pattern 900 searches locations 905, denoted as A0, A1, B0, B1, and / or B2, within the same picture / frame as the current block 901 for a valid candidate motion vector. The candidate list determination pattern 900 may also search co-located blocks 909 for a valid candidate motion vector. The co-located blocks 909 are blocks that are co-located with the current block 901 but are contained in a temporally adjacent picture / frame. The candidate motion vectors may then be arranged in a candidate list 911 in a predetermined examination order. The candidate list 911 is thus a procedurally generated list of indexed candidate motion vectors.

[0104] The candidate list 911 can be used to select a motion vector for performing inter prediction for the current block 901. For example, an encoder can obtain samples of reference blocks pointed to by candidate motion vectors from the candidate list 911. The encoder can then select a candidate motion vector that points to a reference block that most closely matches the current block 901. The index of the selected candidate motion vector can then be encoded to represent the current block 901. In some cases, the candidate motion vector points to a reference block that includes partial reference samples 915. In this case, an interpolation filter 913 can be used to reconstruct complete reference samples 915 to support motion vector selection. The interpolation filter 913 is a filter capable of upsampling a signal. Specifically, the interpolation filter 913 is a filter capable of accepting a partial / lower-quality signal as input and determining an approximation of a more complete / higher-quality signal. Thus, the interpolation filter 913 can be used in certain cases to obtain a complete set of reference samples 915 for use in selecting a reference block for the current block 901, and therefore in selecting a motion vector for encoding the current block 901.

[0105] The above mechanism for coding blocks based on inter prediction by using candidate lists may cause certain errors when subpictures such as subpicture 510 are used. Specifically, problems may arise when the current block 901 is included in the current subpicture, but the motion vector points to a reference block that is at least partially located in an adjacent subpicture. In such a case, the current subpicture may be extracted for presentation without the adjacent subpicture. When this occurs, portions of the reference block in the adjacent subpicture may not be transmitted to the decoder, and therefore the reference block may not be available for decoding the current block 901. When this occurs, the decoder does not have access to sufficient data to decode the current block 901.

[0106] This disclosure provides a mechanism to address this issue. In one example, a flag is used to indicate that the current subpicture should be treated as a picture. This flag can be set to support separate extraction of subpictures. Specifically, when the flag is set, the current subpicture should be coded without reference to data in other subpictures. In this case, the current subpicture is treated like a picture, in that it is coded separately from other subpictures and can be displayed as a separate picture. Therefore, this flag may be indicated as subpic_treated_as_pic_flag[i], where i is the index of the current subpicture. When the flag is set, motion vector candidates (also known as motion vector predictors) obtained from the co-located block 909 include only motion vectors that point within the current subpicture. Any motion vector predictors that point outside the current subpicture are excluded from the candidate list 911. This ensures that motion vectors that point outside the current subpicture are not selected and associated errors are avoided. This example applies particularly to motion vectors from the co-located block 909. Motion vectors from search positions 905 within the same picture / frame may be corrected by different mechanisms, as described below.

[0107] Another example may be used to address the search location 905 when the current subpicture is treated as a picture (e.g., when subpic_treated_as_pic_flag[i] is set). When the current subpicture is treated like a picture, the current subpicture should be extracted without reference to other subpictures. An exemplary mechanism relates to an interpolation filter 913. The interpolation filter 913 can be applied to samples at one location to interpolate (e.g., predict) related samples at other locations. In this example, a motion vector from a coded block at the search location 905 may point to a reference sample 915 outside the current subpicture, as long as the interpolation filter 913 can interpolate such a reference sample 915 based only on the reference sample 915 from the current subpicture. Therefore, this example uses a clipping function that is applied when applying the interpolation filter 913 to motion vector candidates from the search location 905 from the same picture. This clipping function clips data from adjacent sub-pictures, thus eliminating such data as input to the interpolation filter 913 when determining the reference samples 915 pointed to by the motion vector candidates. This approach maintains separation between sub-pictures during encoding to support separate extraction and decoding when the sub-pictures are treated as pictures. The clipping function may be applied to the luma sample bilinear interpolation process, the luma sample 8-tap interpolation filtering process, and / or the chroma sample interpolation process.

[0108] 10 is a block diagram illustrating an exemplary in-loop filter 1000. The in-loop filter 1000 may be used to implement the in-loop filters 225, 325, and / or 425. Furthermore, the in-loop filter 1000 may be applied at the encoder and decoder when performing the method 100. Furthermore, the in-loop filter 1000 may be applied to filter the current block 801 partitioned from the picture 500, which may be coded according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 based on a candidate list generated according to the pattern 900. The in-loop filter 1000 includes a deblocking filter 1043, an SAO filter 1045, and an adaptive loop filter (ALF) 1047. The filters of the in-loop filter 1000 are applied to the reconstructed image block in sequence at the encoder (e.g., before use as a reference block) and at the decoder before display.

[0109] The deblocking filter 1043 is configured to remove block-shaped edges created by block-based inter and intra prediction. The deblocking filter 1043 scans an image portion (e.g., an image slice) for discontinuities in chroma and / or luma values ​​that occur at partition boundaries. The deblocking filter 1043 then applies a smoothing function to block boundaries to remove such discontinuities. The strength of the deblocking filter 1043 may be varied depending on the spatial activity (e.g., variance of luma / chroma components) occurring in areas adjacent to the block boundaries.

[0110] The SAO filter 1045 is configured to remove artifacts related to sample distortion caused by the encoding process. The SAO filter 1045 in the encoder classifies the deblocked samples of the reconstructed image into several categories based on the relative deblocking edge shape and / or direction. An offset is then determined and added to the samples based on the category. The offset is then coded into the bitstream and used by the SAO filter 1045 in the decoder. The SAO filter 1045 removes banding artifacts (bands of values ​​rather than smooth transitions) and ringing artifacts (spurious signals near sharp edges).

[0111] The ALF 1047 is configured at the encoder to compare the reconstructed image with the original image. The ALF 1047 determines coefficients that describe the differences between the reconstructed image and the original image, for example, via a Wiener-based adaptive filter. Such coefficients are coded into the bitstream and used by the ALF 1047 at the decoder to remove the differences between the reconstructed image and the original image.

[0112] Image data filtered by in-loop filter 1000 is output to a picture buffer 1023, which is substantially similar to decoded picture buffer 223, 323, and / or 423. As noted above, deblocking filter 1043, SAO filter 1045, and / or ALF 1047 can be turned off at subpicture and / or tile boundaries by flags such as loop_filter_across_subpic_enabled flag and / or loop_filter_across_tiles_enabled_flag, respectively.

[0113] 11 is a schematic diagram illustrating an exemplary bitstream 1100 including coding tool parameters for supporting decoding of sub-pictures of a picture. For example, the bitstream 1100 can be generated by the codec system 200 and / or the encoder 300 for decoding by the codec system 200 and / or the decoder 400. As another example, the bitstream 1100 may be generated by the encoder in step 109 of the method 100 for use by the decoder in step 111. Furthermore, the bitstream 1100 may include the coded picture 500, the corresponding sub-picture 510, and / or associated coded blocks such as the current block 801 and / or 901, which may be coded according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 based on candidate lists generated according to the pattern 900. The bitstream 1100 may also include parameters for configuring the in-loop filter 1000.

[0114] The bitstream 1100 includes a sequence parameter set (SPS) 1110, multiple picture parameter sets (PPS) 1111, multiple slice headers 1115, and image data 1120. The SPS 1110 includes sequence data common to all pictures in the video sequence included in the bitstream 1100. Such data may include picture size, bit depth, coding tool parameters, bit rate limits, etc. The PPS 1111 includes parameters that apply to the entire picture. Thus, each picture in the video sequence may reference the PPS 1111. While each picture references the PPS 1111, it should be noted that in some instances, a single PPS 1111 may include data for multiple pictures. For example, multiple similar pictures may be coded according to similar parameters. In such cases, a single PPS 1111 may include data for such similar pictures. The PPS 1111 may indicate the coding tools, quantization parameters, offsets, etc. available for slices in the corresponding picture. The slice header 1115 contains parameters specific to each slice in a picture. Thus, there may be one slice header 1115 per slice in a video sequence. The slice header 1115 may include slice type information, a picture order count (POC), a reference picture list, prediction weights, tile entry points, deblocking parameters, etc. It should be noted that the slice header 1115 may also be referred to as a tile group header in some circumstances.

[0115] The image data 1120 includes video data coded according to inter-prediction and / or intra-prediction, as well as corresponding transformed and quantized residual data. For example, a video sequence includes multiple pictures coded as image data. A picture is a single frame of a video sequence and is therefore typically displayed as a single unit when displaying the video sequence. However, sub-pictures may be displayed to implement certain technologies, such as virtual reality and picture-in-picture. Each picture references a PPS 1111. As described above, a picture is divided into sub-pictures, tiles, and / or slices. In some systems, a slice is referred to as a tile group containing tiles. A tile group of slices and / or tiles references a slice header 1115. A slice is further divided into CTUs and / or CTBs. The CTUs / CTBs are further divided into coding blocks based on a coding tree. The coding blocks can then be encoded / decoded according to a prediction mechanism.

[0116] The parameter set in bitstream 1100 includes various data that can be used to implement the examples described herein. To support the implementation of the first example, SPS 1110 of bitstream 1100 includes a subpicture treated as picture flag 1131 for a specified subpicture. In some examples, the subpicture treated as picture flag 1131 is denoted as subpic_treated_as_pic_flag[i], where i is the index of the subpicture associated with the flag. For example, the subpicture treated as picture flag 1131 may be set equal to 1 to specify that the i-th subpicture of each coded picture in the coded video sequence (in image data 1120) is treated as a picture in the decoding process excluding in-loop filtering operations. The subpicture treated as picture flag 1131 may be used when the current subpicture in the current picture is coded according to inter prediction. When the sub-picture treated as picture flag 1131 is set to indicate that the current sub-picture is treated as a picture, the candidate list of candidate motion vectors for the current block can be determined by excluding from the candidate list co-located motion vectors that are contained in the co-located block and point outside the current sub-picture. This ensures that when the current sub-picture is extracted separately from other sub-pictures, motion vectors that point outside the current sub-picture are not selected and related errors are avoided.

[0117] In some examples, the candidate list of motion vectors for the current block is determined according to temporal luma motion vector prediction. For example, when the current block is a luma block of luma samples, the selected current motion vector for the current block is a temporal luma motion vector that points to a reference luma sample in a reference block, and the current block is coded based on the reference luma sample, temporal luma motion vector prediction may be used. In such a case, temporal luma motion vector prediction is performed according to the following: xColBr=xCb+cbWidth; yColBr=yCb+cbHeight; rightBoundaryPos=subpic_treated_as_pic_flag[SubPicIdx]? SubPicRightBoundaryPos:pic_width_in_luma_samples-1; and botBoundaryPos=subpic_treated_as_pic_flag[SubPicIdx]? SubPicBotBoundaryPos:pic_height_in_luma_samples-1 where xColBr and yColBR specify the position of the co-located block, xCb and yCb specify the top-left sample of the current block relative to the top-left sample of the current picture, cbWidth is the width of the current block, cbHeight is the height of the current block, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and pic_width_in_luma_samples is the width of the current picture measured in luma samples. , pic_height_in_luma_samples is the height of the current picture measured in luma samples, botBoundaryPos is the calculated position of the bottom boundary of the subpicture, rightBoundaryPos is the calculated position of the right boundary of the subpicture, SubPicIdx is the index of the subpicture, co-located motion vectors are excluded when yCb>>CtbLog2SizeY is not equal to yColBr>>CtbLog2SizeY, and CtbLog2SizeY indicates the size of the coding tree block.

[0118] The sub-picture treated as a picture flag 1131 may also be used for the implementation of the second example. As in the first example, the sub-picture treated as a picture flag 1131 may be used when the current sub-picture in the current picture is coded according to inter prediction. In this example, a motion vector can be determined for the current block of the sub-picture (e.g., from a candidate list). When the sub-picture treated as a picture flag 1131 is set, a clipping function can be applied to a sample location in a reference block. The sample location is a location within the picture that can contain a single sample containing a luma value and / or a pair of chroma values. An interpolation filter can then be applied when the motion vector points outside the current sub-picture. This clipping function ensures that the interpolation filter does not rely on data from adjacent sub-pictures to maintain separation between sub-pictures to support separate extraction.

[0119] The clipping function can be applied in the luma sample bilinear interpolation process. The luma sample bilinear interpolation process may receive an input including a luma position in whole sample units (xIntL, yIntL). The luma sample bilinear interpolation process outputs a predicted luma sample value (predSampleLXL). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following applies: xInti = Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xIntL+i), and yInti=Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yIntL+i) where subpic_treated_as_pic_flag is a flag that is set to indicate that the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and Clip3 is a clipping function according to

number

[0120] The clipping function can also be applied in the luma sample 8-tap interpolation filtering process. The luma sample 8-tap interpolation filtering process receives an input containing a luma position in whole sample units (xIntL, yIntL). The luma sample bilinear interpolation process outputs a predicted luma sample value (predSampleLXL). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following applies: xInti = Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xIntL+i-3), and yInti=Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yIntL+i-3) where subpic_treated_as_pic_flag is a flag that is set to indicate that the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and Clip3 is as above.

[0121] Clipping functions can also be applied in the chroma sample interpolation process. The chroma sample interpolation process receives input containing chroma positions in whole sample units (xIntC, yIntC). The chroma sample interpolation process outputs predicted chroma sample values ​​(predSampleLXC). Clipping functions are applied to sample positions as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following applies: xInti = Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,xIntC+i), and yInti=Clip3(SubPicTopBoundaryPos / SubHeightC,SubPicBotBoundaryPos / SubHeightC,yIntC+i) where subpic_treated_as_pic_flag is a flag that is set to indicate that the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, SubWidthC and SubHeightC indicate the horizontal and vertical sampling rate ratios between luma samples and chroma samples, and Clip3 is as above.

[0122] The loop filter enable flag 1132 across subpictures in the SPS 1110 may be used for the third example implementation scheme. The loop filter enable flag 1132 across subpictures may be set to control whether filtering is used across a specified subpicture boundary. For example, the loop filter enable flag 1132 across subpictures may be indicated as loop_filter_across_subpic_enabled_flag. The loop filter enable flag 1132 across subpictures may be set to 1 to specify that in-loop filtering operations can be performed across subpicture boundaries, or may be set to 0 to specify that in-loop filtering operations are not performed across subpicture boundaries. Thus, filtering operations may or may not be performed across subpicture boundaries based on the value of the loop filter enable flag 1132 across subpictures. The filtering operations may include applying a deblocking filter 1043, an ALF 1047, and / or an SAO filter 1045. Thus, the filters can be turned off for sub-pictures that are coded for separate extraction, or turned on for sub-pictures that are coded for display as a group.

[0123] A cross-tile loop filter enable flag 1134 in the PPS 1111 may be used for the fourth example implementation scheme. The cross-tile loop filter enable flag 1134 may be set to control whether filtering is used across specified tile boundaries. For example, the cross-tile loop filter enable flag 1134 may be indicated as loop_filter_across_tiles_enabled_flag. The cross-tile loop filter enable flag 1134 may be set to 1 to specify that in-loop filtering operations can be performed across tile boundaries, or may be set to 0 to specify that in-loop filtering operations are not performed across tile boundaries. Thus, a filtering operation may or may not be performed across a specified tile boundary based on the value of the cross-tile loop filter enable flag 1134. The filtering operation may include applying a deblocking filter 1043, an ALF 1047, and / or an SAO filter 1045.

[0124] Subpicture data 1133 in SPS 1110 may be used for the implementation of the fifth example. Subpicture data 1133 may include the width, height, and offset for each subpicture in image data 1120. For example, the width and height of each subpicture may be described in subpicture data 1133 in units of CTB. In some examples, the width and height of a subpicture may be stored in subpicture data 1133 as subpic_width_minus1 and subpic_height_minus1, respectively. Furthermore, the offset of each subpicture may be described in subpicture data 1133 in units of CTU. For example, the offset of each subpicture may be specified as the vertical position and horizontal position of the top-left CTU of the subpicture. Specifically, the offset of a subpicture may be specified as the difference between the top-left CTU of the picture and the top-left CTU of the subpicture. In some examples, the vertical and horizontal positions of the top-left CTU of a subpicture are stored as subpic_ctu_top_left_y and subpic_ctu_top_left_x, respectively, in subpicture data 1133. This example implementation describes subpictures in subpicture data 1133 in terms of CTBs / CTUs rather than tiles. This allows subpictures to be used even when tiles are not used in the corresponding picture / subpicture.

[0125] The subpicture data 1133 in the SPS 1110, the slice address 1136 in the slice header 1115, and the slice subpicture ID 1135 in the slice header 1115 may be used for the implementation of the sixth example. The subpicture data 1133 may be implemented as described in the implementation of the fifth example. The slice address 1136 may include a subpicture-level slice index for the slice (e.g., image data 1120) associated with the slice header 1115. For example, slices are indexed based on the position of the slice within the subpicture rather than based on the position of the slice within the picture. The slice address 1136 may be stored in a slice_address variable. The slice subpicture ID 1135 includes the ID of the subpicture that includes the slice associated with the slice header 1115. Specifically, the slice subpicture ID 1135 may reference a description (e.g., width, height, and offset) of the corresponding subpicture in the subpicture data 1133. The slice subpicture ID 1135 may be stored in the slice_subpic_id variable. Thus, the slice address 1136 is signaled as an index based on the position of the slice within the subpicture indicated by the subpicture ID 1135, as described in the subpicture data 1133. In this way, the position of a slice within a subpicture can be determined even when the subpictures are extracted separately and other subpictures are omitted from the bitstream 1100. This is because this addressing scheme separates the addresses of each subpicture from the other subpictures. Thus, the slice header 1115 does not need to be rewritten when a subpicture is extracted, as would be required in an addressing scheme in which slices are addressed based on the slice's position within the picture. It should be noted that this approach may also be used when the slice is a rectangular / square slice (as opposed to a raster-scan slice). For example, the rect_slice_flag in the PPS 1111 can be set equal to 1 to indicate that the slice is a rectangular / square slice.

[0126] An exemplary implementation of subpictures used in some video coding systems is as follows: Information related to subpictures that may exist in the CVS may be signaled in the SPS. Such signaling may include the following information: The number of subpictures present in each picture in the CVS may be included in the SPS. In the context of the SPS or CVS, co-located subpictures for all access units (AUs) may collectively be referred to as a subpicture sequence. A loop for further specifying information related to the characteristics of each subpicture may also be included in the SPS. Such information may include subpicture identification, subpicture location (e.g., the offset distance between the subpicture's top-left corner luma sample and the picture's top-left corner luma sample), and subpicture size. Furthermore, the SPS may also be used to signal whether each of the subpictures is a motion-constrained subpicture, where a motion-constrained subpicture is a subpicture that includes an MCTS. Profile, tier, and level information for each subpicture may be included in the bitstream, unless such information is otherwise derivable. Such information may be used for profile, tier, and level information for the extracted bitstream created from extracting subpictures from the original bitstream containing the entire picture. The profile and tier of each subpicture may be derived to be the same as the original profile and tier. The level for each subpicture may be explicitly signaled. Such signaling may be present in the loop above. Sequence-level hypothetical reference decoder (HRD) parameters may be signaled in the video usability information (VUI) portion of the SPS for each subpicture (or equivalently, each subpicture sequence).

[0127] When a picture is not partitioned into two or more subpictures, the characteristics of the subpictures (e.g., position, size, etc.) do not need to be signaled in the bitstream, except for the subpicture ID. When a subpicture in a picture in the CVS is extracted, each access unit in the new bitstream may not contain a subpicture. This is because the resulting image data in each AU in the new bitstream is not partitioned into multiple subpictures. Therefore, subpicture characteristics such as position and size may be omitted from the SPS, since such information can be derived from the picture characteristics. However, the subpicture identification is still signaled, since this ID may be referenced by the video coding layer (VCL) NAL units / tile groups included in the extracted subpicture. To reduce resource usage, changing the subpicture ID when extracting a subpicture should be avoided.

[0128] The position (x offset and y offset) of a sub-picture within a picture may be signaled in units of luma samples and may represent the distance between the top-left corner luma sample of the sub-picture and the top-left corner luma sample of the picture. In another example, the position of a sub-picture within a picture may be signaled in units of minimum coding luma block size (MinCbSizeY) and may represent the distance between the top-left corner luma sample of the sub-picture and the top-left corner luma sample of the picture. In another example, the unit of the sub-picture position offset may be explicitly indicated by a syntax element in the parameter set, and the unit may be CtbSizeY, MinCbSizeY, luma sample, or another value. A codec may require that the width of a sub-picture be an integer multiple of the luma CTU size (CtbSizeY) when the right border of the sub-picture does not coincide with the right border of the picture. Similarly, the codec may further require that the subpicture height be an integer multiple of CtbSizeY when the bottom boundary of the subpicture does not coincide with the bottom boundary of the picture. The codec may also require that the subpicture be located at the rightmost position of the picture when the subpicture width is not an integer multiple of the luma CTU size. Similarly, the codec may require that the subpicture be located at the bottommost position of the picture when the subpicture height is not an integer multiple of the luma CTU size. When the subpicture width is signaled in units of luma CTU size and the subpicture width is not an integer multiple of the luma CTU size, the actual width in luma samples may be derived based on the subpicture offset position, the subpicture width in luma CTU size, and the picture width in luma samples. Similarly, when the height of a subpicture is signaled in units of luma CTU size and the height of the subpicture is not an integer multiple of the luma CTU size, the actual height in luma samples can be derived based on the offset position of the subpicture, the height of the subpicture in luma CTU size, and the height of the picture in luma samples.

[0129] For any subpicture, the subpicture ID may be different from the subpicture index. The subpicture index may be the index of the subpicture as signaled in the subpicture loop within the SPS. Alternatively, the subpicture index may be an index assigned in the subpicture raster scan order for the picture. When the value of the subpicture ID of each subpicture is the same as its subpicture index, the subpicture ID may be signaled or derived. When the subpicture ID of each subpicture is different from its subpicture index, the subpicture ID is explicitly signaled. The number of bits for signaling the subpicture ID may be signaled in the same parameter set containing the subpicture characteristics (e.g., in the SPS). Some values ​​for the subpicture ID may be reserved for specific purposes. Such value reservation may be as follows: When a tile group / slice header includes a sub-picture ID to specify which sub-pictures include the tile group, the value 0 may be reserved and not used for the sub-picture to ensure that the first few bits of the tile group / slice header are not all 0 and to avoid generating emulation prevention code. When the sub-pictures of a picture do not cover the entire area of ​​the picture without overlaps and gaps, a value (e.g., value 1) may be reserved for tile groups that are not part of any sub-picture. Alternatively, the sub-picture IDs of the remaining areas may be explicitly signaled. The number of bits for signaling the sub-picture ID may be constrained as follows: The range of values ​​should be sufficient to uniquely identify all sub-pictures in the picture, including the reserved values ​​of the sub-picture ID. For example, the minimum number of bits for the sub-picture ID may be the value of Ceil(Log2(number of sub-pictures in the picture + number of reserved sub-picture IDs).

[0130] The collection of sub-pictures in the loop may be required to cover the entire picture without gaps and without overlaps. When this constraint is applied, there is a flag for each sub-picture to specify whether the sub-picture is a motion-constrained sub-picture, meaning that the sub-picture can be extracted. Alternatively, the collection of sub-pictures may not cover the entire picture. However, there may be no overlap between the sub-pictures of a picture.

[0131] The sub-picture ID may be present immediately after the NAL unit header so that the extractor does not need to understand the rest of the NAL unit bits to assist the sub-picture extraction process. For VCL NAL units, the sub-picture ID may be present in the first bit of the tile group header. For non-VCL NAL units, the following conditions may apply: The sub-picture ID may not need to be present immediately after the NAL unit header for an SPS. For a PPS, when all tile groups of the same picture are constrained to reference the same PPS, the sub-picture ID does not need to be present immediately after the NAL unit header. On the other hand, if tile groups of the same picture are allowed to reference different PPSs, the sub-picture ID may be present in the first bit of the PPS (e.g., immediately after the PPS NAL unit header). In this case, two different tile groups of one picture are not allowed to share the same PPS. Alternatively, when tile groups of the same picture are allowed to reference different PPSs and different tile groups of the same picture are allowed to share the same PPS, the sub-picture ID is not present in the PPS syntax. Alternatively, when tile groups of the same picture are allowed to reference different PPSs and different tile groups of the same picture are allowed to share the same PPS, a list of sub-picture IDs is present in the PPS syntax. The list indicates the sub-pictures to which the PPS applies. For other non-VCL NAL units, if the non-VCL unit applies at or above the picture level (e.g., access unit delimiter, end of sequence, end of bitstream, etc.), the sub-picture ID does not need to be present immediately after its NAL unit header. Otherwise, the sub-picture ID may be present immediately after the NAL unit header.

[0132] Tile partitioning within individual subpictures may be signaled in the PPS, but tile groups within the same picture are allowed to reference different PPSs. In this case, tiles are grouped within each subpicture, not across pictures. Therefore, the concept of tile grouping in this case includes partitioning subpictures into tiles. Alternatively, a Sub-Picture Parameter Set (SPPS) may be used to describe tile partitioning within individual subpictures. The SPPS references the SPS by using a syntax element that references the SPS ID. The SPPS may include a subpicture ID. For the purpose of subpicture extraction, the syntax element that references the subpicture ID is the first syntax element in the SPPS. The SPPS includes a tile structure indicating the number of columns, the number of rows, uniform tile spacing, etc. The SPPS may include a flag to indicate whether the loop filter is valid across the associated subpicture boundary. Alternatively, subpicture characteristics for each subpicture may be signaled in the SPPS instead of the SPS. Tile partitioning within individual subpictures may be signaled with a PPS, but tile groups within the same picture are allowed to reference different PPSs. Once activated, an SPPS may continue throughout a sequence of consecutive AUs in decoding order, but may be deactivated / activated in an AU that is not the start of a CVS. Multiple SPPSs may be active at any point during the decoding process of a single-layer bitstream with multiple subpictures, and an SPPS may be shared by different subpictures of an AU. Alternatively, the SPPS and PPS can be merged into one parameter set. For this to occur, all tile groups included in the same subpicture may be constrained to reference the same parameter set resulting from the merge between the SPPS and the PPS.

[0133] The number of bits used to signal the sub-picture ID may be signaled in the NAL unit header. Such information, when present, aids the sub-picture extraction process in parsing the sub-picture ID value for the start of the NAL unit payload (e.g., the first few bits immediately after the NAL unit header). For such signaling, some of the reserved bits in the NAL unit header may be used to avoid increasing the length of the NAL unit header. The number of bits for such signaling should cover the value of sub-picture-ID-bit-len. For example, four of the seven reserved bits in the NAL unit header of VVC may be used for this purpose.

[0134] When decoding a subpicture, the position of each coding tree block, denoted as the vertical CTB position (xCtb) and horizontal CTB position (yCtb), is adjusted to the actual luma sample position in the picture, not the luma sample position in the subpicture. In this way, extraction of co-located subpictures from each reference picture can be avoided, since everything is decoded as if it were located in the picture, not the subpicture. To adjust the position of the coding tree block, the variables SubpictureXOffset and SubpictureYOffset are derived based on the subpicture position (subpic_x_offset and subpic_y_offset). The values ​​of the variables are added to the values ​​of the x and y coordinates of the luma sample position of each coding tree block in the subpicture, respectively. The subpicture extraction process can be defined as follows: The input to the process includes the target subpicture to be extracted. This can be input in the form of a subpicture ID or a subpicture position. When the input is a subpicture position, the associated subpicture ID can be resolved by analyzing the subpicture information in the SPS. For non-VCL NAL units, the following applies: Syntax elements in the SPS related to picture size and level are updated with the subpicture size and level information. The following non-VCL NAL units are unchanged by the extraction: PPS, access unit delimiter (AUD), end of sequence (EOS), end of bitstream (EOB), and any other non-VCL NAL units applicable at or above the picture level. Remaining non-VCL NAL units whose subpicture ID is not equal to the target subpicture ID are removed. VCL NAL units whose subpicture ID is not equal to the target subpicture ID are also removed.

[0135] The sub-picture nesting SEI message may be used to nest AU-level or sub-picture-level SEI messages for a set of sub-pictures. Data carried in the sub-picture nesting SEI message may include buffering period, picture timing, and non-HRD SEI messages. The syntax and semantics of this SEI message may be as follows: For system operation such as in an omnidirectional media format (OMAF) environment, a set of sub-picture sequences covering a viewport may be requested and decoded by an OMAF player. Therefore, a sequence-level SEI message may carry information about a set of sub-picture sequences that together comprise a rectangular or square picture area. This information can be used by the system and indicates the minimum decoding capability and the bitrate of the set of sub-picture sequences. This information includes the level of a bitstream that contains only the set of sub-picture sequences, the bitrate of the bitstream, and optionally a sub-bitstream extraction process specified for the set of sub-picture sequences.

[0136] The above implementation scheme has several problems. Signaling the picture width and height and / or the subpicture width / height / offset is inefficient. Signaling such information can save more bits. When subpicture size and position information is signaled in the SPS, the PPS includes the tile configuration. Furthermore, the PPS is allowed to be shared by multiple subpictures of the same picture. Therefore, the value ranges for num_tile_columns_minus1 and num_tile_rows_minus1 should be more clearly specified. Furthermore, the semantics of the flag indicating whether a subpicture is motion constrained or not is not clearly specified. The level is mandatory for each subpicture sequence. However, signaling the level of a subpicture is not useful when the subpicture sequence cannot be decoded independently. Furthermore, in some applications, some subpicture sequences should be decoded and rendered together with at least one other subpicture sequence. Therefore, signaling the level for a single such subpicture sequence may not be useful. Furthermore, determining the level value for each sub-picture can be burdensome for the encoder.

[0137] With the introduction of independently decodable subpicture sequences, scenarios requiring independent extraction and decoding of specific regions of a picture may not work based on tile groups. Therefore, explicit signaling of tile group IDs may not be useful. Furthermore, the respective values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag should be the same for all PPSs referenced by the tile group headers of a coded picture. This is because the active PPS should not change within a CVS, and the value of loop_filter_across_tiles_enabled_flag should be the same for all tiles in a picture for tile-based parallel processing. Whether to allow a mix of rectangular and raster scan tile groups within a picture should be clearly specified. Whether to allow subpictures that are part of different pictures and use the same subpicture ID in a CVS to use different tile group modes should also be specified. The derivation process for temporal luma motion vector prediction may not allow treating sub-picture boundaries as picture boundaries in temporal motion vector prediction (TMVP). Furthermore, the luma sample bilinear interpolation process, the luma sample 8-tap interpolation filtering process, and the chroma sample interpolation process may not be configured to treat sub-picture boundaries as picture boundaries in motion compensation. Also, mechanisms for controlling deblocking, SAO, and ALF filtering operations at sub-picture boundaries should be specified.

[0138] With the introduction of independently decodable subpicture sequences, loop_filter_across_tile_groups_enabled_flag may be less useful because turning off in-loop filtering operations for parallel processing purposes may also be satisfied by setting loop_filter_across_tile_groups_enabled_flag equal to 0. Furthermore, turning off in-loop filtering operations to enable independent extraction and decoding of specific regions of a picture may also be satisfied by setting loop_filter_across_sub_pic_enabled_flag equal to 0. Therefore, further specifying a process for turning off in-loop filtering operations across tile group boundaries based on loop_filter_across_tile_groups_enabled_flag unnecessarily burdens the decoder and wastes bits. Furthermore, the above decoding process may not allow for turning off ALF filtering operations across tile boundaries.

[0139] Therefore, this disclosure includes designs for supporting sub-picture-based video coding. A sub-picture is a rectangular or square region within a picture that may or may not be independently decoded using the same decoding process as the picture. The description of the techniques is based on the Versatile Video Coding (VVC) standard. However, the techniques may also be applied to other video codec specifications.

[0140] In some examples, size units are signaled for the picture width and height syntax elements and the list of subpicture width / height / offset_x / offset_y syntax elements. All syntax elements are signaled in the format xxx_minus1. For example, when the size unit is 64 luma samples, a width value of 99 specifies a picture width of 6400 luma samples. The same example applies to others of these syntax elements. In other examples, one or more of the following apply: Size units may be signaled for the picture width and height syntax elements in the format xxx_minus1. Such size units signaled for the list of subpicture width / height / offset_x / offset_y syntax elements may also be in the format xxx_minus1. In other examples, one or more of the following apply: The size units for the list of picture width and subpicture width / offset_x syntax elements may be signaled in the form xxx_minus1. The size units for the list of picture height and subpicture height / offset_y syntax elements may be signaled in the form xxx_minus1. In other examples, one or more of the following apply: The picture width and height syntax elements in the form xxx_minus1 may be signaled in units of minimum coding units. The subpicture width / height / offset_x / offset_y syntax elements in the form xxx_minus1 may be signaled in units of CTUs or CTBs. The subpicture width for each subpicture at the right picture boundary may be derived. The subpicture height for each subpicture at the bottom picture boundary may be derived. All other values ​​of subpicture width / height / offset_x / offset_y may be signaled in the bitstream. In another example, modes for signaling the width and height of sub-pictures and their position within the picture may be added for the case where the sub-pictures have a uniform size.Subpictures have a uniform size when they contain the same subpicture rows and subpicture columns. In this mode, the number of subpicture rows, the number of subpicture columns, the width of each subpicture column, and the height of each subpicture row may all be signaled.

[0141] In another example, the signaling of the subpicture width and height may not be included in the PPS. num_tile_columns_minus1 and num_tile_rows_minus1 should be in the range from 0 to a single integer value, such as 1024. In another example, when a subpicture referencing a PPS has more than one tile, two syntax elements conditioned by a presence flag may be signaled in the PPS. These syntax elements are used to signal the subpicture width and height in units of CTBs and specify the size of all subpictures referencing the PPS.

[0142] In other examples, further information describing individual subpictures may also be signaled. A flag such as sub_pic_treated_as_pic_flag[i] may be signaled for each subpicture sequence to indicate whether the subpictures of the subpicture sequence are treated as pictures in the decoding process for purposes other than in-loop filtering operations. The level to which a subpicture sequence conforms may only be signaled when sub_pic_treated_as_pic_flag[i] is equal to 1. A subpicture sequence is a CVS of subpictures with the same subpicture ID. When sub_pic_treated_as_pic_flag[i] is equal to 1, the level of the subpicture sequence may also be signaled. This can be controlled by a flag for all subpicture sequences or by one flag for each subpicture sequence. In other examples, sub-bitstream extraction may be enabled without modifying the VCL NAL units. This can be achieved by removing the explicit tile group ID signaling from the PPS. The semantics of tile_group_address are specified when rect_tile_group_flag is equal to indicate a rectangular tile group. The tile_group_address may contain a tile group index of a tile group within a tile group in a subpicture.

[0143] In another example, the respective values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag shall be the same in all PPSs referenced by the tile group header of a coded picture. Other PPS syntax elements may be different for different PPSs referenced by the tile group header of a coded picture. The value of single_tile_in_pic_flag may be different for different PPSs referenced by the tile group header of a coded picture. In this way, some pictures in a CVS may have only one tile, while some other pictures in the CVS may have multiple tiles. This also allows some subpictures of a picture (e.g., very large ones) to have multiple tiles, while other subpictures of the same picture (e.g., very small ones) have only one tile.

[0144] In another example, a picture may contain a mix of rectangular / square and raster scan tile groups. Thus, some subpictures of a picture use rectangular / square tile group mode, while other subpictures use raster scan tile group mode. This flexibility is beneficial in bitstream merge scenarios. Alternatively, the constraint may require that all subpictures of a picture use the same tile group mode. Subpictures from different pictures with the same subpicture ID in a CVS may not use different tile group modes. Subpictures from different pictures with the same subpicture ID in a CVS may use different tile group modes.

[0145] In another example, when sub_pic_treated_as_pic_flag[i] for a subpicture is equal to 1, co-located motion vectors for temporal motion vector prediction for the subpicture are constrained to originate from within the boundary of the subpicture. Thus, temporal motion vector prediction for the subpicture is treated as if the subpicture boundary were a picture boundary. Furthermore, to enable treating the subpicture boundary as a picture boundary in motion compensation for subpictures with sub_pic_treated_as_pic_flag[i] equal to 1, clipping operations are specified as part of the luma sample bilinear interpolation process, the luma sample 8-tap interpolation filtering process, and the chroma sample interpolation process.

[0146] In another example, each sub-picture is associated with a signaled flag such as loop_filter_across_sub_pic_enabled_flag. The flag is used to control in-loop filtering operations at the boundaries of the sub-picture and to control filtering operations in the corresponding decoding process. The deblocking filter process may not be applied to coding sub-block edges and transform block edges that coincide with boundaries of sub-pictures for which loop_filter_across_sub_pic_enabled_flag is equal to 0. Alternatively, the deblocking filter process is not applied to coding sub-block edges and transform block edges that coincide with the top or left boundaries of sub-pictures for which loop_filter_across_sub_pic_enabled_flag is equal to 0. Alternatively, the deblocking filter process is not applied to coding sub-block edges and transform block edges that coincide with boundaries of sub-pictures for which sub_pic_treated_as_pic_flag[i] is equal to 1 or 0. Alternatively, the deblocking filter process is not applied to coding sub-block edges and transform block edges that coincide with the top or left boundaries of sub-pictures. A clipping operation may be specified to turn off SAO filtering operations across subpicture boundaries when loop_filter_across_sub_pic_enabled_flag for a subpicture is equal to 0. A clipping operation may be specified to turn off ALF filtering operations across subpicture boundaries when loop_filter_across_sub_pic_enabled_flag is equal to 0 for a subpicture. The loop_filter_across_tile_groups_enabled_flag may also be removed from the PPS. Thus, when loop_filter_across_tiles_enabled_flag is equal to 0, in-loop filtering operations across tile group boundaries that are not subpicture boundaries are not turned off. Loop filter operations may include deblocking, SAO, and ALF.In another example, clipping operation is specified to turn off ALF filtering operation across tile boundaries when loop_filter_across_tiles_enabled_flag for the tile is equal to 0.

[0147] One or more of the above examples may be implemented as follows: A subpicture may be defined as a rectangular or square region of one or more tile groups or slices within a picture. The following divisions of processing elements, i.e., division of each picture into components, division of each component into CTBs, division of each picture into subpictures, division of each subpicture into tile columns within the subpicture, division of each subpicture into tile rows within the subpicture, division of each tile column within a subpicture into tiles, division of each tile row within a subpicture into tiles, and division of each subpicture into tile groups, may form a spatial or component-wise partition.

[0148] The process for the CTB raster and tile scan process within a subpicture may be as follows: A list ColWidth[i], for i in the range 0 to num_tile_columns_minus1, specifying the width of the i-th tile column in units of CTB, may be derived as follows:

number

[0149] The list RowHeight[j], for j in the range 0 to num_tile_rows_minus1 inclusive, specifying the height of the jth tile row in CTB units, is derived as follows:

number

[0150] The list ColBd[i], for i in the range 0 to num_tile_columns_minus1+1 inclusive, specifying the position of the i-th tile column boundary in CTB units, is derived as follows:

number

[0151] The list RowBd[j], for j in the range 0 to num_tile_rows_minus1+1 inclusive, specifying the position of the jth tile row boundary in CTB units, is derived as follows:

number

[0152] The list CtbAddrRsToTs[ctbAddrRs] of ctbAddrRs in the range of 0 to SubPicSizeInCtbsY-1, which specifies the conversion from CTB addresses in the CTB raster scan of the subpicture to CTB addresses in the tile scan of the subpicture, is derived as follows:

number

[0153] The list CtbAddrTsToRs[ctbAddrTs] for ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1, which specifies the conversion from CTB addresses in tile scan to CTB addresses in subpicture CTB raster scan, is derived as follows:

number

[0154] The list TileId[ctbAddrTs] for ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1, which specifies the conversion from CTB addresses to tile IDs in a subpicture tile scan, is derived as follows:

number

[0155] The list NumCtusInTile[tileIdx] for tileIdx in the range of 0 to NumTilesInSubPic - 1, which specifies the conversion from tile index to the number of CTUs within a tile, is derived as follows. [Number]

[0156] The list FirstCtbAddrTs[tileIdx] for tileIdx in the range of 0 to NumTilesInSubPic - 1, which specifies the conversion from tile ID to the CTB address in the tile scan of the first CTB within a tile, is derived as follows. [Number]

[0157] The value of ColumnWidthInLumaSamples[i] that specifies the width of the i-th tile column in luma samples is set equal to ColWidth[i] << CtbLog2SizeY for i in the range of 0 to num_tile_columns_minus1. The value of RowHeightInLumaSamples[j] that specifies the height of the j-th tile row in luma samples is set equal to RowHeight[j] << CtbLog2SizeY for j in the range of 0 to num_tile_rows_minus1.

[0158] The syntax of an exemplary sequence parameter set RBSP is as follows. [Table 1] TIFF0007819281000012.tif35169

[0159] The syntax of an exemplary picture parameter set RBSP is as follows. [Table 2] TIFF0007819281000014.tif34169

[0160] The syntax of an exemplary general tile group header is as follows: [Table 3]

[0161] The syntax of an exemplary coding tree unit is as follows: [Table 4]

[0162] The semantics of an exemplary sequence parameter set RBSP are as follows:

[0163] bit_depth_chroma_minus8 specifies the bit depth of the samples in the chroma array BitDepthC, and the values ​​of the chroma quantization parameter range offset QpBdOffsetC are as follows:

number

[0164] num_sub_pics_minus1 plus 1 specifies the number of subpictures in each coded picture in the CVS. The value of num_sub_pics_minus1 shall be in the range 0 to 1024, inclusive. sub_pic_id_len_minus1 plus 1 specifies the number of bits used to represent the syntax element sub_pic_id[i] in the SPS and the syntax element tile_group_sub_pic_id in the tile group header. The value of sub_pic_id_len_minus1 shall be in the range of Ceil(Log2(num_sub_pic_minus1+1)-1 to 9, inclusive. sub_pic_level_present_flag is set to 1 to specify that the syntax element sub_pic_level_idc[i] may be present. sub_pic_level_present_flag is set to 0 to specify that the syntax element sub_pic_level_idc[i] is not present. sub_pic_id[i] specifies the subpicture ID of the ith subpicture of each coded picture in the CVS. The length of sub_pic_id[i] is sub_pic_id_len_minus1+1 bits.

[0165] sub_pic_treated_as_pic_flag[i] is set equal to 1 to specify that the i-th subpicture of each coded picture in the CVS is treated as a picture in the decoding process, except for in-loop filtering operations. sub_pic_treated_as_pic_flag[i] is set equal to 0 to specify that the i-th subpicture of each coded picture in the CVS is not treated as a picture in the decoding process, except for in-loop filtering operations. sub_pic_level_idc[i] indicates the level to which the i-th subpicture sequence conforms, and the i-th subpicture sequence consists only of VCL NAL units of subpictures with subpicture IDs equal to sub_pic_id[i] in the CVS and their associated non-VCL NAL units. sub_pic_x_offset[i] specifies the horizontal offset, in units of luma samples, of the top-left corner luma sample of the i-th subpicture relative to the top-left corner luma sample of each picture in the CVS. When not present, the value of sub_pic_x_offset[i] is inferred to be equal to 0. sub_pic_y_offset[i] specifies the vertical offset, in units of luma samples, of the top-left luma sample of the ith subpicture relative to the top-left luma sample of each picture in the CVS. When not present, the value of sub_pic_y_offset[i] is inferred to be equal to 0. sub_pic_width_in_luma_samples[i] specifies the width, in units of luma samples, of the ith subpicture of each picture in the CVS. When the sum of sub_pic_x_offset[i] and sub_pic_width_in_luma_samples[i] is less than pic_width_in_luma_samples, the value of sub_pic_width_in_luma_samples[i] shall be an integer multiple of CtbSizeY. When not present, the value of sub_pic_width_in_luma_samples[i] is inferred to be equal to pic_width_in_luma_samples.sub_pic_height_in_luma_samples[i] specifies the height, in units of luma samples, of the i-th subpicture of each picture in the CVS. If the sum of sub_pic_y_offset[i] and sub_pic_height_in_luma_samples[i] is less than pic_height_in_luma_samples, the value of sub_pic_height_in_luma_samples[i] shall be an integer multiple of CtbSizeY. If not present, the value of sub_pic_height_in_luma_samples[i] is inferred to be equal to pic_height_in_luma_samples.

[0166] For bitstream compatibility, the following constraints apply: For any integer values ​​of i and j, when i is equal to j, the values ​​of sub_pic_id[i] and sub_pic_id[j] shall not be the same. For any two subpictures subpicA and subpicB, when the subpicture ID of subpicA is smaller than the subpicture ID of subpicB, any coded tile group NAL unit of subPicA shall follow any coded tile group NAL unit of subPicB in decoding order. The shape of the subpictures shall be such that each subpicture, when decoded, has its entire left and entire top boundaries consisting of the picture boundary or the boundary of a previously decoded subpicture.

[0167] The list SubPicIdx[spId] for spId values ​​equal to sub_pic_id[i] for i in the range 0 to num_sub_pics_minus1, inclusive, that specifies the conversion from subpicture ID to subpicture index, is derived as follows:

number

[0168] log2_max_pic_order_cnt_lsb_minus4 specifies the value of the variable MaxPicOrderCntLsb used in the decoding process for picture order count as follows:

number

[0169] The semantics of an exemplary picture parameter set RBSP are as follows:

[0170] When present, the respective values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag shall be the same for all PPSs referenced by the tile group header of the coded picture. pps_pic_parameter_set_id identifies the PPS referenced by other syntax elements. The value of pps_pic_parameter_set_id shall be in the range from 0 to 63, inclusive. pps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for the active SPS. The value of pps_seq_parameter_set_id shall be in the range from 0 to 15, inclusive. loop_filter_across_sub_pic_enabled_flag shall be set equal to 1 to specify that in-loop filtering operations may be performed across sub-picture boundaries referencing the PPS. loop_filter_across_sub_pic_enabled_flag is set equal to 0 to specify that in-loop filtering operations are not performed across sub-picture boundaries that reference a PPS.

[0171] single_tile_in_sub_pic_flag is set equal to 1 to specify that there is only one tile in each subpicture that references the PPS. single_tile_in_sub_pic_flag is set equal to 0 to specify that there is more than one tile in each subpicture that references the PPS. num_tile_columns_minus1 plus 1 specifies the number of tile columns that partition the subpicture. num_tile_columns_minus1 shall be in the range from 0 to 1024, inclusive. When not present, the value of num_tile_columns_minus1 is inferred to be equal to 0. num_tile_rows_minus1 plus 1 specifies the number of tile rows that partition the subpicture. num_tile_rows_minus1 shall be in the range from 0 to 1024, inclusive. When not present, the value of num_tile_rows_minus1 is inferred to be equal to 0. The variable NumTilesInSubPic is set equal to (num_tile_columns_minus1+1)*(num_tile_rows_minus1+1). When single_tile_in_sub_pic_flag is equal to 0, NumTilesInSubPic shall be greater than 1.

[0172] uniform_tile_spacing_flag is set equal to 1 to specify that tile column boundaries, and similarly tile row boundaries, are uniformly distributed among the subpictures. uniform_tile_spacing_flag is set equal to 0 to specify that tile column boundaries, and similarly tile row boundaries, are not uniformly distributed among the subpictures, but are explicitly signaled using the syntax elements tile_column_width_minus1[i] and tile_row_height_minus1[i]. When not present, the value of uniform_tile_spacing_flag is inferred to be equal to 1. tile_column_width_minus1[i] plus 1 specifies the width of the i-th tile column in units of CTBs. tile_row_height_minus1[i] plus 1 specifies the height of the i-th tile row in units of CTBs. single_tile_per_tile_group is set equal to 1 to specify that each tile group referencing this PPS contains one tile. single_tile_per_tile_group is set equal to 0 to specify that the tile group that references this PPS may contain more than one tile.

[0173] rect_tile_group_flag is set equal to 0 to specify that the tiles in each tile group of the subpicture are in raster scan order and that tile group information is not signaled in the PPS. rect_tile_group_flag is set equal to 1 to specify that the tiles in each tile group cover a rectangular or square area of ​​the subpicture and that tile group information is signaled in the PPS. When single_tile_per_tile_group_flag is set to 1, rect_tile_group_flag is inferred to be equal to 1. num_tile_groups_in_sub_pic_minus1 plus 1 specifies the number of tile groups in each subpicture that refer to the PPS. The value of num_tile_groups_in_sub_pic_minus1 shall be in the range from 0 to NumTilesInSubPic-1, inclusive. When not present and single_tile_per_tile_group_flag is equal to 1, the value of num_tile_groups_in_sub_pic_minus1 is inferred to be equal to NumTilesInSubPic-1.

[0174] top_left_tile_idx[i] specifies the tile index of the tile located in the top-left corner of the i-th tile group of the subpicture. The value of top_left_tile_idx[i] shall not be equal to the value of top_left_tile_idx[j] for any i not equal to j. When not present, the value of top_left_tile_idx[i] is inferred to be equal to i. The length of the top_left_tile_idx[i] syntax element is Ceil(Log2(NumTilesInSubPic)) bits. bottom_right_tile_idx[i] specifies the tile index of the tile located in the bottom right corner of the i-th tile group of the subpicture. When single_tile_per_tile_group_flag is set to 1, bottom_right_tile_idx[i] is inferred to be equal to top_left_tile_idx[i]. The length of the bottom_right_tile_idx[i] syntax element is Ceil(Log2(NumTilesInSubPic)) bits.

[0175] It is a bitstream conformance requirement that any particular tile be contained in only one tile group. The variable NumTilesInTileGroup[i], which specifies the number of tiles in the i-th tile group of a subpicture, and related variables are derived as follows:

number

[0176] loop_filter_across_tiles_enabled_flag is set equal to 1 to specify that in-loop filtering operations may be performed across tile boundaries in subpictures that reference a PPS. loop_filter_across_tiles_enabled_flag is set equal to 0 to specify that in-loop filtering operations are not performed across tile boundaries in subpictures that reference a PPS. In-loop filtering operations include deblocking filter, sample adaptive offset filter, and adaptive loop filter operations. When not present, the value of loop_filter_across_tiles_enabled_flag is inferred to be equal to 1. num_ref_idx_default_active_minus1[i] plus 1 specifies the estimated value of the variable NumRefIdxActive[0] for P or B tile groups with num_ref_idx_active_override_flag equal to 0 when i is equal to 0, and specifies the estimated value of NumRefIdxActive[1] for B tile groups with num_ref_idx_active_override_flag equal to 0 when i is equal to 1. The value of num_ref_idx_default_active_minus1[i] shall be in the range 0 to 14, inclusive.

[0177] The semantics of an exemplary general tile group header are as follows: When present, the respective values ​​of the tile group header syntax elements tile_group_pic_order_cnt_lsb and tile_group_temporal_mvp_enabled_flag shall be the same in all tile group headers of a coded picture. When present, the value of tile_group_pic_parameter_set_id shall be the same in all tile group headers of a coded sub-picture. tile_group_pic_parameter_set_id specifies the value of pps_pic_parameter_set_id for the PPS in use. The value of tile_group_pic_parameter_set_id shall be in the range from 0 to 63, inclusive. It is a bitstream conformance requirement that the value of TemporalId of the current picture shall be greater than or equal to the value of TemporalId of each PPS referenced by the tile group of the current picture. tile_group_sub_pic_id identifies the sub-picture to which the tile group belongs. The length of tile_group_sub_pic_id is sub_pic_id_len_minus1 + 1 bits. The value of tile_group_sub_pic_id shall be the same for all tile group headers of a coded sub-picture.

[0178] The variables SubPicWidthInCtbsY, SubPicHeightInCtbsY and SubPicSizeInCtbsY are derived as follows:

number

[0179] The following variables: a list ColWidth[i] for i in the range 0 to num_tile_columns_minus1 inclusive, specifying the width of the ith tile column in CTBs; a list RowHeight[j] for j in the range 0 to num_tile_rows_minus1 inclusive, specifying the height of the jth tile row in CTBs; a list ColBd[i] for i in the range 0 to num_tile_columns_minus1+1 inclusive, specifying the position of the ith tile column boundary in CTBs; A list RowBd[j] for j in the range 0 to num_tile_rows_minus1+1, specifying the position of the tile row boundary. A list CtbAddrRsToTs[ctbAddrRs] for ctbAddrRs in the range 0 to SubPicSizeInCtbsY-1, specifying the conversion from the CTB address in the subpicture CTB raster scan to the CTB address in the subpicture tile scan. Specifies the conversion from CTB addresses to tile IDs in the subpicture tile scan. CtbAddrTsToRs[ctbAddrTs] is a list of ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1 inclusive. TileId[ctbAddrTs] is a list of ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1 inclusive. Specifies the conversion from tile indexes to the number of CTUs in a tile. A list NumCtusInTile[tileIdx] for the range tileIdx, a list FirstCtbAddrTs[tileIdx] for the range tileIdx from 0 to NumTilesInSubPic-1 inclusive that specifies the conversion from tile ID to the CTB address in the tile scan of the first CTB in the tile, a list ColumnWidthInLumaSamples[i] for i in the range 0 to num_tile_columns_minus1 inclusive that specifies the width of the i-th tile column in units of luma samples,and a list RowHeightInLumaSamples[j], for j in the range 0 to num_tile_rows_minus1, inclusive, specifying the height of the jth tile row in units of luma samples, derived by invoking the CTB raster and tile scan conversion process.

[0180] The values ​​of ColumnWidthInLumaSamples[i] for i in the range of 0 to num_tile_columns_minus1, inclusive, and RowHeightInLumaSamples[j] for j in the range of 0 to num_tile_rows_minus1, inclusive, shall all be greater than 0. The variables SubPicLeftBoundaryPos, SubPicTopBoundaryPos, SubPicRightBoundaryPos, and SubPicBotBoundaryPos are derived as follows.

number

[0181] For each tile with index i=0.NumTilesInSubPic-1 in the current subpicture, the variables TileLeftBoundaryPos[i], TileTopBoundaryPos[i], TileRightBoundaryPos[i] and TileBotBoundaryPos[i] are derived as follows:

number

[0182] tile_group_address specifies the tile address of the first tile in the tile group. When not present, the value of tile_group_address is inferred to be equal to 0. If rect_tile_group_flag is equal to 0, the following applies: the tile address is a tile ID, the length of tile_group_address is Ceil(Log2(NumTilesInSubPic)) bits, and the value of tile_group_address shall be in the range from 0 to NumTilesInSubPic-1 inclusive. Otherwise (if rect_tile_group_flag is equal to 1), the following applies: the tile address is a tile group index of a tile group within a tile group in a subpicture, the length of tile_group_address is Ceil(Log2(num_tile_groups_in_sub_pic_minus1+1)) bits, and the value of tile_group_address shall be in the range from 0 to num_tile_groups_in_sub_pic_minus1 inclusive.

[0183] It is a requirement for bitstream conformance that the following constraints apply: The value of tile_group_address shall not be equal to the value of tile_group_address of any other coded tile group NAL unit of the same coded subpicture. The tile groups of a subpicture shall be in increasing order of their tile_group_address values. The shape of the tile groups of a subpicture shall be such that each tile, when decoded, shall have its entire left and entire top boundaries consisting of either the subpicture boundary or the boundaries of a previously decoded tile.

[0184] num_tiles_in_tile_group_minus1, when present, specifies the number of tiles in the tile group minus 1. The value of num_tiles_in_tile_group_minus1 shall be in the range 0 to NumTilesInSubPic-1 inclusive. When not present, the value of num_tiles_in_tile_group_minus1 is inferred to be equal to 0. The variables NumTilesInCurrTileGroup, which specifies the number of tiles in the current tile group, and TgTileIdx[i], which specifies the tile index of the i-th tile in the current tile group, are derived as follows:

number

[0185] An exemplary derivation process for temporal luma motion vector prediction is as follows: The variables mvLXCol and availableFlagLXCol are derived as follows: If tile_group_temporal_mvp_enabled_flag is equal to 0, then both components of mvLXCol are set equal to 0, and availableFlagLXCol is set equal to 0. Otherwise (tile_group_temporal_mvp_enabled_flag is equal to 1), the following sequence of steps applies: The bottom right co-located motion vector, and the bottom and right boundary sample positions are derived as follows:

number

[0186] An exemplary luma sample bilinear interpolation process is as follows: The luma position in whole sample units (xInti, yInti) is derived as follows, for i=0..1: If sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following applies:

number

number

[0187] An exemplary luma sample 8-tap interpolation filtering process is as follows: The luma position in whole sample units (xInti, yInti) is derived as follows, for i=0..7: If sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following applies:

number

number

[0188] An exemplary chroma sample interpolation process is as follows: The variable xOffset is set equal to (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC. The chroma position in whole sample units (xInti, yInti), for i=0..3, is derived as follows:

[0189] If sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following is true:

number

number

[0190] An exemplary deblocking filter process is as follows: The deblocking filter process is applied to all coding sub-block and transform block edges of a picture except for the following types of edges: edges at picture boundaries, edges that coincide with sub-picture boundaries with loop_filter_across_sub_pic_enabled_flag equal to 0, edges that coincide with tile boundaries with loop_filter_across_tiles_enabled_flag equal to 0, edges that coincide with the top or left boundary of or within a tile group with tile_group_deblocking_filter_disabled_flag equal to 1, edges that do not correspond to an 8x8 sample grid boundary of the component being considered, edges in chroma components where both sides of the edge use inter prediction, edges of chroma transform blocks that are not edges of the associated transform unit, and edges that cross a luma transform block of a coding unit with an IntraSubPartitionsSplit value not equal to ISP_NO_SPLIT.

[0191] An exemplary deblocking filter process in one direction is as follows. For each coding unit having a coding block width log2CbW, a coding block height log2CbH, and the position (xCb, yCb) of the top-left sample of the coding block, when edgeType is equal to EDGE_VER and xCb%8 is equal to 0, or when edgeType is equal to EDGE_HOR and yCb%8 is equal to 0, the edge is filtered by the steps in the following order. The coding block width nCbW is set equal to 1<<log2CbW, and the coding block height nCbH is set equal to 1<<log2CbH. The variable filterEdgeFlag is derived as follows. When edgeType is equal to EDGE_VER and one or more of the following conditions are true, filterEdgeFlag is set equal to 0. The left boundary of the current coding block is the left boundary of the picture. The left boundary of the current coding block is the left or right boundary of the subpicture and loop_filter_across_sub_pic_enabled_flag is equal to 0. The left boundary of the current coding block is the left boundary of the tile and loop_filter_across_tiles_enabled_flag is equal to 0. Otherwise, when edgeType is equal to EDGE_HOR and one or more of the following conditions are true, the variable filterEdgeFlag is set equal to 0. The top boundary of the current luma coding block is the top boundary of the picture. The top boundary of the current coding block is the top or bottom boundary of the subpicture and loop_filter_across_sub_pic_enabled_flag is equal to 0. The top boundary of the current coding block is the top boundary of the tile and loop_filter_across_tiles_enabled_flag is equal to 0. Otherwise, filterEdgeFlag is set equal to 1.

[0192] An exemplary CTB modification process is as follows: For all sample positions (xSi, ySj) and (xYi, yYj), for i = 0.. nCtbSw-1 and j = 0.. nCtbSh-1, depending on the values ​​of pcm_loop_filter_disabled_flag, pcm_flag[xYi][yYj], and cu_transquant_bypass_flag of the coding unit containing the coding block covering recPicture[xSi][ySj], the following is true: If one or more of the following conditions are true for all sample positions (xSik', ySjk') and (xYik', yYjk'), for k = 0.. 1, then edgeIdx is set equal to 0: The sample at position (xSik', ySjk') is outside the picture boundary ...), The sample at position (xSik', ySjk') belongs to a different subpicture, and loop_filter_across_sub_pic_enabled_flag in the tile group to which sample recPicture[xSi][ySj] belongs is equal to 0. loop_filter_across_tiles_enabled_flag is equal to 0, and the sample at position (xSik', ySjk') belongs to a different tile.

[0193] An example coding tree block filtering process for luma samples is as follows: To derive a filtered reconstructed luma sample alfPictureL[x][y], each reconstructed luma sample in the current luma coding tree block recPictureL[x][y] is filtered with x,y=0..CtbSizeY-1 as follows: Each position (hx,vy) of the corresponding luma sample (x,y) in a given array of luma samples recPicture is derived as follows: If loop_filter_across_tiles_enabled_flag for the tile tileA that contains the luma sample at position (hx,vy) is equal to 0, then the following applies, where variable tileIdx is the tile index of tileA:

number

number

number

[0194] An exemplary derivation process for the ALF transpose and filter index for a luma sample is as follows: Each position (hx,vy) of a corresponding luma sample (x,y) in a given array of luma samples, recPicture, is derived as follows: If loop_filter_across_tiles_enabled_flag is equal to 0 for the tile tileA that contains the luma sample at position (hx,vy), then, let tileIdx be the tile index of tileA, then the following holds:

number

number

number

[0195] An example coding tree block filtering process for chroma samples is as follows: To derive a filtered reconstructed chroma sample alfPicture[x][y], each reconstructed chroma sample in the current chroma coding tree block recPicture[x][y] is filtered by x,y=0..ctbSizeC-1 as follows: Each position (hx,vy) of the corresponding chroma sample (x,y) in a given array of chroma samples recPicture is derived as follows: If loop_filter_across_tiles_enabled_flag is equal to 0 for the tile tileA that contains the chroma sample at position (hx,vy), then the following holds, where tileIdx is the tile index of tileA:

number

number

number

[0196] The sum of the variables is derived as follows:

number

number

[0197] FIG. 12 is a schematic diagram of an exemplary video coding device 1200. The video coding device 1200 is suitable for implementing the disclosed examples / embodiments as described herein. The video coding device 1200 includes a downstream port 1220, an upstream port 1250, and / or a transceiver unit (Tx / Rx) 1210, each including a transmitter and / or a receiver for communicating data upstream and / or downstream over a network. The video coding device 1200 also includes a processor 1230 including a logic unit and / or central processing unit (CPU) for processing data, and a memory 1232 for storing data. The video coding device 1200 may also include electrical, optical-to-electrical (OE), electrical-to-optical (EO), and / or wireless communication components coupled to the upstream port 1250 and / or the downstream port 1220 for communicating data via an electrical, optical, or wireless communication network. Video coding device 1200 may also include input and / or output (I / O) devices 1260 for communicating data to and from a user. I / O devices 1260 may include output devices such as a display for displaying video data, speakers for outputting audio data, etc. I / O devices 1260 may also include input devices such as a keyboard, mouse, trackball, etc., and / or corresponding interfaces for interacting with such output devices.

[0198] The processor 1230 is implemented in hardware and software. The processor 1230 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 1230 communicates with the downstream port 1220, the Tx / Rx 1210, the upstream port 1250, and the memory 1232. The processor 1230 includes a coding module 1214. The coding module 1214 may implement the disclosed embodiments described herein, such as methods 100, 1300, and 1400, using the in-loop filter 1000, the bitstream 1100, the picture 500, and / or the current block 801 and / or 901, which may be coded according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 based on the candidate list generated according to the pattern 900. Coding module 1214 may also implement any other method / mechanism described herein. Furthermore, coding module 1214 may implement codec system 200, encoder 300, and / or decoder 400. For example, coding module 1214 may implement the first, second, third, fourth, fifth, and / or sixth example implementations described above. Thus, coding module 1214 enables video coding device 1200 to provide additional functionality and / or coding efficiency when coding video data. Thus, coding module 1214 improves the functionality of video coding device 1200 and addresses issues specific to video coding techniques. Furthermore, coding module 1214 effects transformation of video coding device 1200 into a different state.Alternatively, the coding module 1214 may be implemented as instructions stored in memory 1232 and executed by the processor 1230 (eg, as a computer program product stored on a non-transitory medium).

[0199] Memory 1232 may include one or more memory types such as a disk, a tape drive, a solid-state drive, read only memory (ROM), random access memory (RAM), flash memory, ternary content-addressable memory (TCAM), static random-access memory (SRAM), etc. Memory 1232 stores such programs when such programs are selected for execution and may also be used as an overflow data storage device to store instructions and data read during program execution.

[0200] 13 is a flowchart of an example method 1300 for encoding a video sequence into a bitstream, such as bitstream 1100, by addressing a slice, such as slice 515, relative to a subpicture, such as subpicture 510. Method 1300 may be used by an encoder, such as codec system 200, encoder 300, and / or video coding device 1200, when performing method 100 for encoding current block 801 and / or 901 according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 using in-loop filter 1000 and / or candidate lists generated according to pattern 900.

[0201] Method 1300 may begin when an encoder receives a video sequence including multiple pictures and decides to encode the video sequence into a bitstream based on, for example, user input. In step 1301, the encoder partitions the pictures into sub-pictures and the sub-pictures into slices.

[0202] In step 1303, the encoder determines a slice address for the slice. Specifically, the slice address includes a sub-picture-level slice index for the slice when the slice is a rectangular or square slice. The sub-picture-level index is an index based on the position of the slice within the sub-picture, not the position of the slice within the picture. Additionally, the encoder determines a sub-picture ID for the sub-picture that contains the slice. In some examples, when rect_slice_flag is set equal to 1, the slice may be a rectangular or square slice. rect_slice_flag may be coded in the SPS and / or PPS.

[0203] In step 1305, the encoder encodes the slice address and the subpicture ID into the bitstream. For example, the encoder may encode a slice header into the bitstream. The slice header includes parameters associated with the slice and used to encode / decode the slice. For example, a slice may reference its slice header. The slice address and the subpicture ID may be encoded in the slice header in the bitstream. In some examples, the slice address is encoded in a slice_address variable, and the subpicture ID is encoded in a slice_subpic_id variable.

[0204] In step 1307, the encoder encodes slices of the subpicture into the bitstream. The encoder may also encode entire subpictures and / or entire pictures in the bitstream. In step 1309, the encoder may also encode an SPS into the bitstream. For example, the encoder may encode descriptions of various subpictures into the SPS. A subpicture ID may reference a description of the associated subpicture in the SPS. In some examples, the subpicture description includes the subpicture width in units of CTBs, the subpicture height in units of CTBs, and the subpicture offset in units of CTBs, specified as the vertical position of the subpicture's top-left CTU and the horizontal position of the subpicture's top-left CTU. In step 1311, the encoder stores the bitstream for communication to the decoder.

[0205] 14 is a flowchart of an example method 1400 of decoding a video sequence from a bitstream, such as bitstream 1100, when a slice, such as slice 515, is addressed relative to a subpicture, such as subpicture 510. Method 1400 may be used by a decoder, such as codec system 200, decoder 400, and / or video coding device 1200, when performing method 100 for decoding current block 801 and / or 901 according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 on a candidate list generated by using in-loop filter 1000 and / or according to pattern 900.

[0206] Method 1400 may begin when a decoder begins receiving a bitstream of coded data representing a video sequence, for example, as a result of method 1300. In step 1401, the decoder receives the bitstream. The bitstream may include an SPS, a PPS, slice headers, and sub-pictures that include slices. In some examples, the bitstream may also include an entire picture that includes multiple sub-pictures.

[0207] In step 1403, the decoder obtains a slice address. The slice address includes a sub-picture level slice index for the slice when the slice is a rectangular or square slice. The sub-picture level index is an index based on the position of the slice in the sub-picture, not based on the position of the slice in the picture. The sub-picture level index is an index based on the position of the slice in the sub-picture, not based on the position of the slice in the picture. Furthermore, the decoder may obtain a sub-picture ID for the sub-picture that contains the slice. For example, a slice header may be associated with the slice and may include parameters used to encode / decode the slice. For example, a slice may reference a slice header. The slice address and sub-picture ID may be obtained from a slice header in the bitstream. In some examples, the slice address is encoded in a slice_address variable, and the sub-picture ID is encoded in a slice_subpic_id variable. In some examples, a slice may be a rectangular or square slice when rect_slice_flag is set equal to 1. The rect_slice_flag may be obtained from the SPS and / or PPS. In step 1405, the decoder may obtain a description of the subpicture from the SPS. For example, the subpicture ID may reference the associated subpicture and thus the description of the subpicture in the SPS. In some examples, the description of the subpicture includes the width of the subpicture in units of CTBs, the height of the subpicture in units of CTBs, and the offset of the subpicture in CTBs specified as the vertical position of the top-left CTU of the subpicture and the horizontal position of the top-left CTU of the subpicture. In step 1407, the decoder may locate a slice within the subpicture based on the slice address, the subpicture ID, and the description of the subpicture from the SPS. In step 1409, the decoder decodes the slice of the subpicture based on the slice address. The decoder can then forward the slice for display as part of the decoded video sequence.

[0208] 15 is a schematic diagram of an example system 1500 for coding a video sequence of images in a bitstream, such as bitstream 1100, when a slice, such as slice 515, is addressed relative to a subpicture, such as subpicture 510. System 1500 may be implemented by an encoder and decoder, such as codec system 200, encoder 300, decoder 400, and / or video coding device 1200. Furthermore, system 1500 may be used when implementing methods 100, 1300, and / or 1400 for coding current block 801 and / or 901 according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 using in-loop filter 1000 and / or candidate lists generated according to pattern 900.

[0209] The system 1500 includes a video encoder 1502. The video encoder 1502 includes a partition module 1505 for partitioning a subpicture into slices. The video encoder 1502 further includes a determination module 1506 for determining a slice address including a subpicture-level slice index of a slice when the slice is a rectangular or square slice. The video encoder 1502 further includes an encoding module 1507 for encoding the slice address into a bitstream. Further, the encoding module 1507 is for encoding the slice of the subpicture into the bitstream. The video encoder 1502 further includes a storage module 1508 for storing the bitstream for communication to a decoder. The video encoder 1502 further includes a transmission module 1509 for transmitting the bitstream to a video decoder 1510. The video encoder 1502 may be further configured to perform any of the steps of the method 1300.

[0210] The system 1500 also includes a video decoder 1510. The video decoder 1510 includes a receiving module 1511 for receiving a bitstream including a sub-picture including a slice. The video decoder 1510 further includes an obtaining module 1512 for obtaining a slice address including a sub-picture level slice index of the slice when the slice is a rectangular or square slice. The video decoder 1510 further includes a decoding module 1513 for decoding the slice of the sub-picture based on the slice address. The video decoder 1510 further includes a transfer module 1515 for transferring the slice for display as part of the decoded video sequence. The video decoder 1510 may be further configured to perform any of the steps of the method 1400.

[0211] A first component is directly coupled to a second component when there are no intervening components, other than lines, traces, or other media between the first and second components. A first component is indirectly coupled to a second component when there are intervening components, other than lines, traces, or other media between the first and second components. The term "coupled" and variations thereof include both directly coupled and indirectly coupled. The use of the term "about," unless otherwise stated, refers to a range that includes ±10% of the subsequent number.

[0212] It should also be understood that the steps of the exemplary methods described herein do not necessarily have to be performed in the order described, and the order of steps of such methods should be understood to be merely exemplary. Likewise, additional steps may be included in such methods, and certain steps may be omitted or combined in methods consistent with various embodiments of the present disclosure.

[0213] While several embodiments are provided in this disclosure, it will be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The examples should be considered illustrative and not limiting, and the intention is not to be limited to the details provided herein. For example, various elements or components may be combined or integrated into other systems, or certain features may be omitted or not implemented.

[0214] Furthermore, techniques, systems, subsystems, and methods described and illustrated in various embodiments as separate or distinct may be combined or integrated with other systems, components, techniques, or methods without departing from the scope of the present disclosure. Other examples of variations, substitutions, and alterations will be ascertainable by those skilled in the art and may be made without departing from the spirit and scope of the present disclosure.

Claims

1. 1. A method implemented in a decoder, comprising: receiving a bitstream including coded data of a picture and a syntax element, the picture including a plurality of sub-pictures, the syntax element specifying the number of the plurality of sub-pictures included in the picture, and a sub-picture of the plurality of sub-pictures including a tile group; obtaining a tile group address from the bitstream, where when the tile group is a rectangular tile group, the tile group address includes a sub-picture level tile group index of the tile group; decoding the tile group of the subpicture based on the tile group address; A method comprising:

2. obtaining a sub-picture identifier (ID) of the sub-picture that includes the tile group; decoding the tile group of the subpicture based on the tile group address, The method of claim 1 , comprising decoding the tile group of the subpicture based on the tile group address and the subpicture ID.

3. The method of claim 1 or 2, wherein the bitstream further comprises a tile group header associated with the tile group, and the tile group address and the sub-picture ID are obtained from the tile group header.

4. 1. A method implemented in an encoder, comprising: partitioning a picture into a plurality of sub-pictures; partitioning a sub-picture of the plurality of sub-pictures into tile groups; encoding a tile group address into a bitstream, where when the tile group is a rectangular tile group, the tile group address includes a sub-picture level tile group index of the tile group; encoding a syntax element into the bitstream, the syntax element specifying the number of the plurality of sub-pictures to be included in the picture; encoding the tile group into the bitstream; A method comprising:

5. The method of claim 4 , further comprising encoding into the bitstream a subpicture identifier (ID) of the subpicture that includes the tile group.

6. The method of claim 4 or 5, wherein the bitstream further comprises a tile group header associated with the tile group, and the tile group address and the sub-picture ID are encoded in the tile group header.

7. 1. A video coding device, comprising: A video coding device comprising a processor and a memory coupled to the processor, the processor and the memory configured to perform the method of any one of claims 1 to 6.

8. A device, a storage medium storing a bitstream, the bitstream including coded data of a picture and syntax elements, the picture including a plurality of sub-pictures, the syntax elements specifying the number of the plurality of sub-pictures included in the picture, and a sub-picture of the plurality of sub-pictures including a tile group; a processor configured to generate the bitstream; Including, The device, wherein the bitstream further includes a tile group address, and when the tile group is a rectangular tile group, the tile group address includes a sub-picture level tile group index of the tile group.

9. The device of claim 8 , further comprising a transmitter configured to transmit the bitstream.

10. 1. A bitstream storage method comprising: generating a bitstream, the bitstream including coded data of a picture and a syntax element, the picture including a plurality of sub-pictures, the syntax element specifying the number of the plurality of sub-pictures included in the picture, and a sub-picture of the plurality of sub-pictures including a tile group; storing the bitstream on a storage medium; Including, The method, wherein the bitstream further includes a tile group address, and when the tile group is a rectangular tile group, the tile group address includes a sub-picture level tile group index of the tile group.

11. The method of claim 10 further comprising transmitting the bitstream on request.

12. A computer program which, when executed on a computer or a processor, causes said computer or said processor to carry out a method according to any one of claims 1 to 6 or 10 to 11.

13. A computer-readable storage medium containing computer-executable instructions, comprising: A computer-readable storage medium, the computer-executable instructions, when executed by a video coding device, causing the video coding device to perform the method of any one of claims 1-6 or 10-11.

14. A decoder comprising: a receiving unit configured to receive a bitstream including coded data of a picture and a syntax element, the picture including a plurality of sub-pictures, the syntax element specifying a number of the plurality of sub-pictures included in the picture, a sub-picture of the plurality of sub-pictures including a tile group; an acquisition unit configured to acquire a tile group address from the bitstream, where when the tile group is a rectangular tile group, the tile group address includes a sub-picture level tile group index of the tile group; a decoding unit configured to decode the tile group of the subpicture based on the tile group address; A decoder containing

15. 1. An encoder comprising: a partitioning unit configured to partition a picture into a plurality of sub-pictures and partition a sub-picture of the plurality of sub-pictures into a tile group; encoding a tile group address into the bitstream, where when the tile group is a rectangular tile group, the tile group address includes a sub-picture level tile group index of the tile group; encoding a syntax element into the bitstream, the syntax element specifying the number of the plurality of sub-pictures included in the picture; Encoding the tile group into the bitstream The coding unit is configured as follows: Encoder including.

Images