Encoder, decoder and corresponding method

Abstract

To provide a video coding mechanism.SOLUTION: A mechanism includes receiving a bit stream including a picture including a sub-picture. The width of the sub-picture of a unit of a coating tree block (CTB) and the height of the sub-picture of the unit of the CTB are acquired from a bit stream. The coding block of the sub-picture is decoded on the basis of the width of the sub-picture and the height of the sub-picture. The coding block of the sub-picture is transferred so as to be displayed as a part of a decoding video sequence.SELECTED DRAWING: Figure 5D

Description

[Technical field]

[0001] [Technical field] FIELD This disclosure relates generally to video coding, and more particularly to coding sub-pictures of a picture in video coding. [Background technology]

[0002] The amount of video data required to render even a relatively short video can be substantial, which can create difficulties when the data is streamed or communicated across communication networks with limited bandwidth capacity. Therefore, video data is typically compressed before being communicated across modern telecommunication networks. The size of the video can also be an issue when the video is stored on a storage device, since memory resources may be limited. Video compression devices often use software and / or hardware at the source to code the video data before transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompression device, which decodes the video data. Due to limited network resources and the ever-increasing demand for higher video quality, improved compression and decompression techniques that improve compression ratios with little or no sacrifice in image quality are desirable. Summary of the Invention

[0003] In an embodiment, the disclosure includes a method implemented in a decoder, the method including: receiving, by a receiver of the decoder, a bitstream including a picture including a subpicture; obtaining, by a processor of the decoder, from the bitstream, a subpicture width in units of coding tree blocks (CTBs) and a subpicture height in units of CTBs; and decoding, by the processor, a coding block of the subpicture based on the subpicture width and the subpicture height. A video coding system may partition a picture into subpictures. This allows different subpictures to be treated differently when coding video. For example, subpictures can be extracted and displayed separately, resized independently based on application level changes, etc. In some cases, subpictures may be created by partitioning a picture into tiles and assigning tiles to subpictures. Some video coding systems describe subpicture boundaries in terms of tiles included in the subpicture. However, a tiling scheme may not be used in some pictures. Thus, such boundary description may restrict the use of subpictures to pictures that use tiles. The present disclosure includes a mechanism for signaling sub-picture boundaries in terms of CTBs and / or CTUs. Specifically, the width and height of a sub-picture can be signaled in units of CTBs. Also, the location of the top-left CTU of the sub-picture can be signaled as an offset from the top-left CTU of the picture as measured in CTBs. The CTU and CTB sizes may be set to predefined values. Thus, signaling sub-picture dimensions and locations in terms of CTBs and CTUs provides sufficient information for a decoder to position the sub-picture for display. This allows the sub-picture to be used even when tiles are not used. Also, this signaling mechanism avoids complexity and can be coded using relatively few bits. Thus, the present example provides additional functionality to video codecs by allowing sub-pictures to be used independently of tiles.Furthermore, the present embodiment increases coding efficiency and therefore reduces the use of processor, memory and / or network resources in the encoder and / or decoder.

[0004] Optionally, in any of the above aspects, another implementation method of the aspect provides further comprising: a step of obtaining, by a processor, a sub-picture offset in units of a CTB from the bitstream; and a step of positioning, by the processor, the sub-picture relative to the picture based on the sub-picture offset.

[0005] Optionally, in any of the above aspects, another implementation of the aspect provides that the offset of a subpicture is specified as the vertical position of the subpicture's top-left coding tree unit (CTU) and the horizontal position of the subpicture's top-left CTU.

[0006] Optionally, in any of the above aspects, another implementation of the aspect provides that the offset of the sub-picture is further specified as a difference between a top-left CTU of the picture and a top-left CTU of the sub-picture.

[0007] Optionally, in any of the above aspects, another implementation of the aspect provides that the vertical position of the top-left CTU of the subpicture is stored in the bitstream as subpic_ctu_top_left_y, and the vertical position of the top-left CTU of the subpicture is stored in the bitstream as subpic_ctu_top_left_x.

[0008] Optionally, in any of the above aspects, another implementation of the aspect provides that the width of the subpicture is stored in the bitstream as subpic_width_minus1 and the height of the subpicture is stored in the bitstream as subpic_height_minus1.

[0009] Optionally, in any of the above aspects, another realization manner of the aspect provides that the width, height and offset of the sub-picture are obtained from a sequence parameter set (SPS) in the bitstream.

[0010] In an embodiment, the disclosure includes a method implemented in an encoder, the method including: partitioning, by a processor of the encoder, a picture into sub-pictures; determining, by the processor, a sub-picture width and a sub-picture height; encoding, by the processor, the sub-picture width and the sub-picture height in units of CTBs into a bitstream; encoding, by the processor, coding blocks of the sub-picture into the bitstream; and storing, by a memory coupled to the processor, the bitstream for communication to a decoder. A video coding system may partition a picture into sub-pictures. This allows different sub-pictures to be treated differently when coding video. For example, sub-pictures can be extracted and displayed separately, resized independently based on application level changes, etc. In some cases, sub-pictures may be created by partitioning a picture into tiles and assigning the tiles to sub-pictures. Some video coding systems describe sub-picture boundaries in terms of tiles included in the sub-picture. However, a tiling scheme may not be used in some pictures. Thus, such boundary description may restrict the use of sub-pictures to pictures that use tiles. This disclosure includes a mechanism for signaling sub-picture boundaries in terms of CTBs and / or CTUs. Specifically, the width and height of a sub-picture can be signaled in units of CTBs. Also, the location of the top-left CTU of the sub-picture can be signaled as an offset from the top-left CTU of the picture as measured in CTBs. The CTU and CTB sizes may be set to predefined values. Thus, signaling sub-picture dimensions and locations in terms of CTBs and CTUs provides sufficient information for a decoder to position the sub-picture for display. This allows sub-pictures to be used even when tiles are not used. Also, this signaling mechanism avoids complexity and can be coded using a relatively small number of bits.Thus, the present example provides additional functionality to video codecs by allowing sub-pictures to be used independently of tiles, and further increases coding efficiency and therefore reduces the use of processor, memory and / or network resources in the encoder and / or decoder.

[0011] Optionally, in any of the above aspects, another implementation method of the aspect provides further comprising: determining, by a processor, a sub-picture offset in units of CTBs; and encoding, by the processor, the sub-picture offset in units of CTBs into a bitstream.

[0012] Optionally, in any of the above aspects, another implementation of the aspect provides that the offset of a subpicture is specified as the vertical position of the top-left CTU of the subpicture and the horizontal position of the top-left CTU of the subpicture.

[0013] Optionally, in any of the above aspects, another implementation of the aspect provides that the offset of the sub-picture is further specified as a difference between a top-left CTU of the picture and a top-left CTU of the sub-picture.

[0014] Optionally, in any of the above aspects, another implementation of the aspect provides that the vertical position of the top-left CTU of the subpicture is coded into the bitstream as subpic_ctu_top_left_y, and the vertical position of the top-left CTU of the subpicture is coded into the bitstream as subpic_ctu_top_left_x.

[0015] Optionally, in any of the above aspects, another implementation of the aspect provides that the width of the subpicture is coded into the bitstream as subpic_width_minus1 and the height of the subpicture is coded into the bitstream as subpic_height_minus1.

[0016] Optionally, in any of the above aspects, another implementation of the aspect provides that the width, height and offset of the sub-picture are coded into the bitstream in the SPS.

[0017] In an embodiment, the present disclosure includes a video coding device including a processor, a receiver coupled to the processor, a memory coupled to the processor, and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform the method of any of the above aspects.

[0018] In an embodiment, the present disclosure includes a non-transitory computer-readable medium including a computer program product for use by a video coding device, the computer program product including computer-executable instructions stored on the non-transitory computer-readable medium that, when executed by a processor, causes the video coding device to perform a method of any of the above aspects.

[0019] In an embodiment, the present disclosure includes a decoder including a receiving means for receiving a bitstream including a picture including a subpicture, an obtaining means for obtaining a subpicture width in units of CTBs and a subpicture height in units of CTBs from the bitstream, a decoding means for decoding a subpicture coding block based on the subpicture width and the subpicture height, and a transferring means for transferring the subpicture coding block for display as part of a decoded video sequence.

[0020] Optionally, in any of the above aspects, another implementation of the aspect provides that the decoder is further configured to perform the method of any of the above aspects.

[0021] In an embodiment, the present disclosure includes an encoder including partitioning means for partitioning a picture into sub-pictures, determining means for determining a sub-picture width and a sub-picture height, encoding means for encoding the sub-picture width and the sub-picture height in units of CTBs into a bitstream and encoding coding blocks of the sub-picture into the bitstream, and storage means for storing the bitstream for communication to a decoder.

[0022] Optionally, in any of the above aspects, another implementation of the aspect provides that the encoder is further configured to perform the method of any of the above aspects.

[0023] For purposes of clarity, any one of the above embodiments may be combined with any one or more of the other above embodiments to create new embodiments within the scope of the present disclosure.

[0024] These and other features will be more clearly understood from the following detailed description considered in conjunction with the accompanying drawings and claims. [Brief description of the drawings]

[0025] For a more complete understanding of the present disclosure, reference is now made to the following brief description taken in conjunction with the accompanying drawings and detailed description, where like reference characters represent like parts. [Figure 1] 4 is a flowchart of an exemplary method for coding a video signal. [Diagram 2] 1 is a schematic diagram of an example coding and decoding (codec) system for video coding. [Diagram 3] FIG. 1 is a schematic diagram illustrating an exemplary video encoder. [Figure 4] FIG. 2 is a schematic diagram illustrating an exemplary video decoder. [Figure 5A] FIG. 2 is a schematic diagram illustrating an exemplary picture partitioned into sub-pictures. [Figure 5B]FIG. 2 is a schematic diagram illustrating an exemplary sub-picture partitioned into slices. [Figure 5C] FIG. 2 is a schematic diagram illustrating an example slice partitioned into tiles. [Figure 5D] 1 is a schematic diagram illustrating an example slice partitioned into coding tree units (CTUs). [Figure 6] FIG. 1 is a schematic diagram illustrating an example of unidirectional inter prediction. [Figure 7] FIG. 1 is a schematic diagram illustrating an example of bidirectional inter prediction. [Figure 8] FIG. 2 is a schematic diagram illustrating an example of coding a current block based on candidate motion vectors from neighboring coded blocks. [Figure 9] FIG. 2 is a schematic diagram illustrating an example pattern for determining a candidate list of motion vectors. [Figure 10] FIG. 2 is a block diagram illustrating an example in-loop filter. [Figure 11] 1 is a schematic diagram illustrating an example bitstream including coding tool parameters to support decoding sub-pictures of a picture. [Figure 12] 1 is a schematic diagram of an example video coding device. [Figure 13] 1 is a flowchart of an exemplary method for encoding a video sequence including sub-pictures into a bitstream. [Figure 14] 1 is a flowchart of an exemplary method for decoding a video sequence including sub-pictures from a bitstream. [Figure 15] 1 is a schematic diagram of an example system for coding a video sequence of images including sub-pictures in a bitstream. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] First, while exemplary implementations of one or more embodiments are provided below, it should be understood that the disclosed systems and / or methods may be implemented using any number of technologies currently known or in existence. The present disclosure should in no way be limited to the exemplary implementations, drawings and technologies set forth below, including the exemplary designs and implementations illustrated and described herein, but may be modified within the scope of the appended claims, together with the full scope of equivalents thereof.

[0027] The following abbreviations are used: Adaptive Loop Filter (ALF), Coding Tree Block (CTB), Coding Tree Unit (CTU), Coding Unit (CU), Coded Video Sequence (CVS), Joint Video Experts Team (JVET), Motion-Constrained Tile Set (MCTS), Maximum Transfer Unit (MTU), Network Abstraction Layer (NAL), Picture Order Count (POC), Raw Byte Sequence Payload (RBSP), Sample Adaptive Offset (SAO), Sequence Parameter Set (SPS), Temporal Motion Vector Prediction (TMVP), Versatile Video Coding (VVC), and Working Draft (WG). WD) is used here.

[0028] Many video compression techniques can be used to reduce the size of a video file with minimal data loss. For example, video compression techniques can include performing spatial (e.g., intra-picture) prediction and / or temporal (e.g., inter-picture) prediction to reduce or remove data redundancy in a video sequence. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks, which may also be referred to as tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are coded using spatial prediction with respect to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded unidirectionally predicted (P) or bidirectionally predicted (B) slice of a picture may be coded using spatial prediction with respect to reference samples in neighboring blocks in the same picture or temporal prediction with respect to reference samples in other reference pictures. A picture may be referred to as a frame and / or an image, and a reference picture may be referred to as a reference frame and / or a reference image. Spatial or temporal prediction results in a predictive block that represents an image block. The residual data represents pixel differences between the original image block and the predictive block. Thus, an inter-coded block is coded according to a motion vector that points to a block of reference samples that form the predictive block, and the residual data that indicates the difference between the block to be coded and the predictive block. An intra-coded block is coded according to an intra-coding mode and the residual data. For further compression, the residual data may be transformed from the pixel domain to a transform domain. These result in residual transform coefficients, which may be quantized. The quantized transform coefficients may first be arranged in a two-dimensional array. The quantized transform coefficients may be scanned to generate a one-dimensional vector of transform coefficients.Entropy coding may be applied to achieve even greater compression, and such video compression techniques are described in more detail below.

[0029] To ensure that the encoded video can be accurately decoded, the video is encoded and decoded according to a corresponding video coding standard, including International Telecommunication Union (ITU) Standardization Sector (ITU-T) H.261, International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) Motion Picture Experts Group (MPEG)-1 Part 2, Advanced Video Coding (AVC), also known as ITU-T H.262 or ISO / IEC MPEG-2 Part 2, ITU-T H.263, ISO / IEC MPEG-4 Part 2, ITU-T H.264 or ISO / IEC MPEG-4 Part 10, and High Efficiency Video Coding (HEVC), also known as ITU-T H.265 or MPEG-H Part 2. AVC includes extensions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding plus Depth (MVC+D) and three dimensional (3D) AVC (3D-AVC). HEVC includes extensions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC) and 3D HEVC (3D-HEVC).The ITU-T and ISO / IEC joint video experts team (JVET) has begun developing a video coding standard called Versatile Video Coding (VVC). VVC is contained in a Working Draft (WD), which includes JVET-M1001-v6, which provides an algorithmic description, an encoder-side description of VVC WD, and reference software.

[0030] To code a video image, the image is first partitioned and the partitions are coded into a bitstream. Various picture partitioning schemes are available. For example, an image can be partitioned into normal slices, dependent slices, tiles, and / or according to Wavefront Parallel Processing (WPP). For simplicity, HEVC restricts the encoder to use only normal slices, dependent slices, tiles, WPP, and combinations thereof when partitioning slices into groups of CTBs for video coding. Such partitioning can be applied to support Maximum Transfer Unit (MTU) size matching, parallel processing, and reduced end-to-end delay. The MTU indicates the maximum amount of data that can be transmitted in a single packet. If a packet payload exceeds the MTU, the payload is split into two packets through a process called fragmentation.

[0031] A normal slice, also simply called a slice, is a partitioned portion of an image that can be rendered independently of other normal slices in the same picture, despite some interdependencies due to loop filtering operations. Each normal slice is encapsulated in its own Network Abstraction Layer (NAL) unit for transmission. Furthermore, intra-picture prediction (intra-sample prediction, motion information prediction, coding mode prediction) and entropy coding dependencies across slice boundaries may be disabled to support independent rendering. Such independent rendering supports parallelization. For example, normal slice-based parallelization uses minimal inter-processor or inter-core communication. However, because each normal slice is independent, each slice is associated with an individual slice header. The use of normal slices can incur substantial coding overhead due to the bit cost of a slice header for each slice and due to the lack of prediction across slice boundaries. Furthermore, normal slices may be used to support matching for MTU size requirements. Specifically, because regular slices can be encapsulated in individual NAL units and coded independently, each regular slice should be smaller than the MTU in the MTU scheme to avoid splitting the slice into multiple packets. Thus, the goals of parallelization and MTU size matching may impose conflicting requirements on the slice layout within a picture.

[0032] Dependent slices are similar to normal slices, but have shortened slice headers and allow for partitioning of picture treeblock boundaries without breaking intra-picture prediction. Dependent slices therefore allow normal slices to be fragmented into multiple NAL units, which provides reduced end-to-end delay by allowing portions of a normal slice to be sent before the encoding of the entire normal slice is completed.

[0033] A tile is a partitioned portion of an image created by horizontal and vertical boundaries that create columns and rows of tiles. Tiles may be coded in raster scan order (right to left and top to bottom). The scan order of the CTBs is local within the tile. Thus, the CTB in the first tile is coded in raster scan order before proceeding to the CTB in the next tile. As with normal slices, tiles break intra-picture prediction dependencies and entropy decoding dependencies. However, tiles may not be included in individual NAL units, and therefore tiles may not be used for MTU size matching. Each tile may be processed by one processor / core, and inter-processor / inter-core communication used for intra-picture prediction between processing units that decode adjacent tiles may be limited to conveying a shared slice header (when adjacent tiles are in the same slice) and performing loop filtering related sharing of reconstructed samples and metadata. When more than one tile is included in a slice, the entry point byte offset for each tile other than the first entry point offset in the slice may be signaled in the slice header. For each slice and tile, at least one of the following conditions should be satisfied: 1) all coded tree blocks in a slice belong to the same tile, and 2) all coded tree blocks in a tile belong to the same slice.

[0034] In WPP, a picture is partitioned into a single row of CTBs. The entropy decoding and prediction mechanism may use data from CTBs in other rows. Parallel processing is possible through parallel decoding of CTB rows. For example, the current row may be decoded in parallel with the previous row. However, the decoding of the current row is delayed from the decoding process of the previous row by two CTBs. This delay ensures that data related to the CTB above the current CTB and the CTB to the right above in the current row are available before the current CTB is coded. This approach appears as a wavefront when represented graphically. This staggered start allows parallelization on up to as many processors / cores as the picture contains CTB rows. Since intra-picture prediction between adjacent treeblock rows in a picture is allowed, inter-processor / inter-core communication to enable intra-picture prediction can be substantial. WPP partitioning takes into account the NAL unit size. Therefore, WPP does not support MTU size matching. However, to achieve MTU size matching as desired, regular slices can be used for WPP with certain coding overhead.

[0035] A tile may also include a motion constrained tile set. A motion constrained tile set (MCTS) is a tile set designed such that associated motion vectors are constrained to all sample positions in the MCTS and fractional sample positions that require only all sample positions in the MCTS for interpolation. Furthermore, the use of motion vector candidates for temporal motion vector prediction derived from blocks outside the MCTS is prohibited. In this way, each MCTS may be decoded independently without the presence of tiles not included in the MCTS. A supplemental enhancement information (SEI) message for a temporal MCTS may indicate the presence of an MCTS in a bitstream and may be used to signal the MCTS. The SEI message for an MCTS provides supplemental information that can be used in an MCTS sub-bitstream extraction (specified as part of the semantics of the SEI message) to generate a conforming bitstream for the MCTS set. The information includes multiple extraction information sets, each defining multiple MCTS sets, including raw bytes sequence payload (RBSP) bytes of replacement video parameter set (VPS), sequence parameter set (SPS) and picture parameter set (PPS) to be used during the MCTS sub-bitstream extraction process. When extracting a sub-bitstream according to the MCTS sub-bitstream extraction process, the parameter sets (VPS, SPS and PPS) may be rewritten or replaced, and the slice header may be updated since one or all of the slice address related syntax elements (including first_slice_segment_in_pic_flag and slice_segment_address) may use different values ​​in the extracted sub-bitstream.

[0036] A picture may also be partitioned into one or more sub-pictures. Partitioning a picture into sub-pictures may allow different portions of a picture to be treated differently from a coding perspective. For example, a sub-picture may be extracted and displayed without extracting other sub-pictures. As other examples, different sub-pictures may be displayed at different resolutions, may be repositioned relative to one another (e.g., in a videoconferencing application), or may be coded as separate pictures even though the sub-pictures together contain data from a common picture.

[0037] An exemplary implementation of sub-pictures is as follows: A picture can be partitioned into one or more sub-pictures. A sub-picture is a rectangular or square set of slices / tile groups starting with the slice / tile group with address equal to 0. Each sub-picture may refer to a different PPS and therefore each sub-picture may use a different partitioning mechanism. Sub-pictures may be treated like pictures in the decoding process. The current reference picture used to decode the current sub-picture may be generated by extracting a region co-located with the current sub-picture from a reference picture in the decoded picture buffer. The extracted region may be a decoded sub-picture, and therefore inter-prediction may be performed between sub-pictures of the same size and location in the picture. A tile group may be a sequence of tiles in the tile raster scan of a sub-picture. The following may be derived to determine the location of a sub-picture in a picture: Each sub-picture may be included in the next unoccupied position in the CTU raster scan order in a picture that is large enough to fit the sub-picture within the picture boundary.

[0038] The sub-picture schemes used by various video coding systems include various problems that reduce coding efficiency and / or functionality. This disclosure includes various solutions to such problems. In a first exemplary problem, inter prediction may be performed according to one of several inter prediction modes. A particular inter prediction mode generates a candidate list of motion vector predictors at both the encoder and the decoder. This allows the encoder to signal a motion vector by signaling an index from the candidate list instead of signaling the entire motion vector. Furthermore, some systems encode sub-pictures for independent extraction. This allows the current sub-picture to be decoded and displayed without decoding information from other sub-pictures. This may result in errors when a motion vector that points outside the sub-picture is used. The reason for this is that the data pointed to by the motion vector may not be decoded and therefore may not be available.

[0039] Thus, in a first example, a flag is disclosed herein that indicates that a sub-picture should be treated as a picture. The flag is set to support separate extraction of sub-pictures. When the flag is set, motion vector predictors obtained from co-located blocks include only motion vectors that point into the sub-picture. Any motion vector predictors that point outside the sub-picture are excluded. This ensures that motion vectors that point outside the sub-picture are not selected and related errors are avoided. Co-located blocks are blocks from pictures different from the current picture. Motion vector predictors from blocks in the current picture (non-co-located blocks) may point outside the sub-picture, since other processes such as an interpolation filter may prevent errors for such motion vector predictors. Thus, this example provides additional functionality to video encoders / decoders (codecs) by preventing errors when performing sub-picture extraction.

[0040] In a second example, a flag is disclosed herein that indicates that a subpicture should be treated as a picture. When a current subpicture is treated like a picture, the current subpicture should be extracted without reference to other subpictures. Specifically, this example uses a clipping function that is applied when applying an interpolation filter. This clipping function ensures that the interpolation filter does not rely on data from adjacent subpictures to maintain separation between subpictures to support separate extraction. Thus, the clipping function is applied when the flag is set and the motion vector points outside the current subpicture. The interpolation filter is then applied to the result of the clipping function. Thus, this example provides further functionality to the video codec by preventing errors when performing subpicture extraction. Thus, the first and second examples address the first exemplary problem.

[0041] In a second example problem, a video coding system partitions a picture into sub-pictures, slices, tiles, and / or coding tree units, which are then partitioned into blocks. Such blocks are then encoded for transmission towards a decoder. Decoding such blocks may result in a decoded image that includes various types of noise. To correct such problems, the video coding system may apply various filters across block boundaries. These filters may remove blocking, quantization noise, and other undesirable coding artifacts. As noted above, some systems encode sub-pictures for independent extraction. This allows a current sub-picture to be decoded and displayed without decoding information from other sub-pictures. In such systems, sub-pictures may be partitioned into blocks for encoding. Thus, block boundaries along sub-picture edges may align with sub-picture boundaries. In some cases, block boundaries may also align with tile boundaries. Filters may be applied across such block boundaries, and thus across sub-picture and / or tile boundaries. This can lead to errors when the current subpicture is extracted independently, since the filtering process may behave in an unexpected way when data from adjacent subpictures is not available.

[0042] In a third example, a flag is disclosed herein that controls filtering at the sub-picture level. When the flag is set for a sub-picture, the filter can be applied across sub-picture boundaries. When the flag is not set, the filter is not applied across sub-picture boundaries. In this manner, the filter can be turned off for sub-pictures that are coded for separate extraction, or turned on for sub-pictures that are coded for display as a group. Thus, this example provides additional functionality to video codecs by preventing filter-related errors when performing sub-picture extraction.

[0043] In a fourth example, a flag is disclosed herein that can be set to control filtering at the tile level. When the flag is set for a tile, the filter can be applied across the tile boundary. When the flag is not set, the filter is not applied across the tile boundary. In this manner, the filter can be turned off or on for use at the tile boundary (e.g., while continuing to filter the interior of the tile). Thus, this example provides further functionality to the video codec by supporting selective filtering across tile boundaries. Thus, the third and fourth examples address the second exemplary problem.

[0044] In a third example problem, a video coding system may partition a picture into sub-pictures. This allows different sub-pictures to be treated differently when coding the video. For example, sub-pictures can be extracted and displayed separately, resized independently based on application-level changes, etc. In some cases, sub-pictures may be created by partitioning a picture into tiles and assigning the tiles to sub-pictures. Some video coding systems describe sub-picture boundaries in terms of tiles included in the sub-picture. However, a tiling scheme may not be used in some pictures. Thus, such boundary descriptions may limit the use of sub-pictures to pictures that use tiles.

[0045] In a fifth example, a mechanism for signaling sub-picture boundaries in terms of CTBs and / or CTUs is disclosed herein. Specifically, the width and height of the sub-picture can be signaled in units of CTBs. Also, the location of the top-left CTU of the sub-picture can be signaled as an offset from the top-left CTU of the picture as measured in CTBs. The CTU and CTB sizes may be set to predefined values. Thus, signaling the sub-picture dimensions and location in terms of CTBs and CTUs provides sufficient information for a decoder to position the sub-picture for display. This allows the sub-picture to be used even when tiles are not used. Also, this signaling mechanism avoids complexity and can be coded using relatively few bits. Thus, this example provides further functionality to video codecs by allowing the sub-picture to be used independently of tiles. Furthermore, this example increases coding efficiency and thus reduces the use of processor, memory and / or network resources in the encoder and / or decoder. Thus, the fifth example addresses the third exemplary problem.

[0046] In a fourth exemplary problem, a picture may be partitioned into multiple slices for encoding. In some video coding systems, slices are addressed based on their location relative to the picture. Still other video coding systems use the concept of sub-pictures. As noted above, sub-pictures may be treated differently from other sub-pictures from a coding perspective. For example, sub-pictures may be extracted and displayed independently of other sub-pictures. In such cases, slice addresses generated based on picture location may not work properly since a significant number of possible slice addresses are omitted. Some video coding systems address this problem by dynamically rewriting slice headers upon request to change slice addresses to support sub-picture extraction. Such a process may be resource intensive, since it may occur every time a user requests to view a sub-picture.

[0047] In a sixth example, a slice addressed to a sub-picture containing the slice is disclosed herein. For example, a slice header may include a sub-picture identifier (ID) and an address of each slice included in the sub-picture. Furthermore, a sequence parameter set (SPS) may include dimensions of the sub-picture that may be referenced by the sub-picture ID. Thus, when separate extraction of a sub-picture is requested, the slice header does not need to be rewritten. The slice header and the SPS include sufficient information to support positioning a slice within a sub-picture for display. Thus, this example increases coding efficiency and / or avoids redundant rewriting of slice headers, thus reducing the use of processor, memory and / or network resources in the encoder and / or decoder. Thus, the sixth example addresses the fourth exemplary problem.

[0048] 1 is a flow chart of an exemplary operational method 100 of coding a video signal. Specifically, a video signal is encoded in an encoder. The encoding process compresses the video signal by using various mechanisms to reduce the video file size. The smaller file size allows the compressed video file to be transmitted towards a user while reducing the associated bandwidth overhead. A decoder then decodes the compressed video file to recreate the original video signal for display to an end user. The decoding process generally mirrors the encoding process to allow the decoder to recreate the video signal consistently.

[0049] In step 101, a video signal is input to the encoder. For example, the video signal may be an uncompressed video file stored in memory. As another example, the video file may be captured by a video capture device such as a video camera and encoded to support live streaming of the video. The video file may include both audio and video components. The video component includes a series of image frames that, when viewed in sequence, give the visual impression of motion. The frames include pixels represented in terms of light, referred to herein as luma components (or luma samples), and colors, referred to herein as chroma components (or color samples). In some examples, the frames may also include depth values ​​to support three-dimensional displays.

[0050] In step 103, the video is partitioned into blocks. Partitioning involves subdividing the pixels in each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H Part 2), a frame may first be partitioned into coding tree units (CTUs), which are blocks of a given size (e.g., 64 pixels by 64 pixels). The CTUs contain both luma and chroma samples. The coding tree may be used to partition the CTUs into blocks and then recursively subdivide the blocks until a configuration is achieved that supports further encoding. For example, the luma component of a frame may be subdivided until each block contains relatively uniform illumination values. Furthermore, the chroma component of a frame may be subdivided until each block contains relatively uniform color values. Thus, the partitioning mechanism varies depending on the content of the video frame.

[0051] In step 105, various compression mechanisms are used to compress the image blocks partitioned in step 103. For example, inter-prediction and / or intra-prediction may be used. Inter-prediction is designed to take advantage of the fact that objects in a common scene tend to appear in consecutive frames. Thus, a block representing an object in a reference frame does not need to be described repeatedly in adjacent frames. In particular, an object such as a table may remain in a constant position across multiple frames. Thus, the table is described once and adjacent frames can be referenced back to the reference frame. A pattern matching mechanism may be used to match objects across multiple frames. Furthermore, a moving object may be represented across multiple frames, for example, due to object motion or camera motion. As a specific example, a video may show a car moving across the screen across multiple frames. To describe such motion, a motion vector may be used. A motion vector is a two-dimensional vector that provides an offset from the coordinates of the object in the frame to the coordinates of the object in the reference frame. Thus, inter-prediction may encode an image block in a current frame as a set of motion vectors that indicate an offset from a corresponding block in a reference frame.

[0052] Intra prediction codes blocks within a common frame. Intra prediction takes advantage of the fact that luma and chroma components tend to cluster within a frame. For example, green fragments in a part of a tree tend to be located adjacent to similar green fragments. Intra prediction uses multiple directional prediction modes (e.g., 33 in HEVC), planar mode and direct current (DC) mode. The directional modes indicate that the current block is similar / same as the samples of neighboring blocks in the corresponding direction. The planar mode indicates that a series of blocks (e.g., a plane) along a row / column can be interpolated based on the neighboring blocks at the end of the row. The planar mode essentially indicates a smooth transition of light / color across the row / column by using a relatively constant slope in changing the values. The DC mode is used for boundary smoothing and indicates that the block is similar / same as the average value associated with the samples of all neighboring blocks associated with the angular direction of the directional prediction mode. Thus, an intra prediction block can represent the image block as various related prediction mode values ​​instead of the actual value. Additionally, inter-prediction blocks can represent image blocks as motion vector values ​​instead of actual values. In either case, the prediction block may not exactly represent the image block in some cases. Any differences are stored in a residual block. To further compress the file, a transform may be applied to the residual block.

[0053] In step 107, various filtering techniques may be applied. In HEVC, filters are applied according to an in-loop filtering scheme. The block-based prediction described above may result in the generation of blocky images at the decoder. Furthermore, the block-based prediction scheme may encode a block and then reconstruct the encoded block for later use as a reference block. The in-loop filtering scheme repeatedly applies noise suppression filters, deblocking filters, adaptive loop filters and sample adaptive offset (SAO) filters to the block / frame. These filters mitigate such blocking artifacts so that the encoded file can be accurately reconstructed. Furthermore, these filters mitigate artifacts in the reconstructed reference block so that the artifacts are less likely to create further artifacts in subsequent blocks that are encoded based on the reconstructed reference block.

[0054] Once the video signal has been segmented, compressed and filtered, the resulting data is coded into a bitstream in step 109. The bitstream includes the above data and any signaling data desired to support proper video signal reproduction at a decoder. For example, such data may include segmentation data, prediction data, residual blocks, and various flags that provide coding instructions to the decoder. The bitstream may be stored in memory for transmission to a decoder upon request. The bitstream may also be broadcast and / or multicast to multiple decoders. Creation of the bitstream is an iterative process. Thus, steps 101, 103, 105, 107 and 109 may occur sequentially and / or simultaneously across many frames and blocks. The order shown in FIG. 1 is presented for clarity and ease of explanation and is not intended to limit the video coding process to a particular order.

[0055] The decoder receives the bitstream and starts the decoding process in step 111. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into corresponding syntax and video data. In step 111, the decoder uses syntax data from the bitstream to determine the partition of the frame. The partition should match the result of the block partition in step 103. We now describe the entropy coding / decoding used in step 111. The encoder makes many choices during the compression process, such as selecting a block partitioning scheme from several possible choices based on the spatial positioning of values ​​in the input image. Signaling the exact choice may use multiple bins. As used herein, a bin is a binary value (e.g., a bit value that can change depending on the context) that is treated as a variable. Entropy coding allows the encoder to discard any options that are obviously infeasible in a particular case, leaving a set of acceptable options. Each acceptable option is then assigned a codeword. The length of the codeword is based on the number of acceptable options (e.g., one bin for two options, two bins for three to four options, etc.). The encoder then encodes a codeword for the selected option. This scheme reduces the size of the codeword because it is desirable for the codeword to be large enough to uniquely indicate a selection from a small subset of permissible options, as opposed to uniquely indicating a selection from a potentially large set of all possible options. The decoder then decodes the selection by determining the set of permissible options in a similar manner as the encoder. By determining the set of permissible options, the decoder can read the codeword and determine the selection made by the encoder.

[0056] In step 113, the decoder performs block decoding. Specifically, the decoder uses an inverse transform to generate a residual block. Then, the decoder uses the residual block and a corresponding prediction block to reconstruct an image block according to the partition. The prediction block may include both intra-prediction blocks and inter-prediction blocks as generated in the encoder in step 105. Then, the reconstructed image block is positioned in a frame of the reconstructed video signal according to the partition data determined in step 111. The syntax for step 113 may also be signaled in the bitstream via entropy coding as described above.

[0057] In step 115, filtering is performed on the frames of the reconstructed video signal in a manner similar to step 107 in the encoder. For example, a noise suppression filter, a deblocking filter, an adaptive loop filter, and an SAO filter may be applied to the frames to remove blocking artifacts. Once the frames have been filtered, the video signal can be output to a display in step 117 for viewing by an end user.

[0058] 2 is a schematic diagram of an exemplary coding and decoding (codec) system 200 for video coding. Specifically, the codec system 200 provides functionality to support the implementation of the operational method 100. The codec system 200 is generalized to show components used in both an encoder and a decoder. The codec system 200 receives and segments a video signal, as described with respect to steps 101 and 103 in the operational method 100, which results in a segmented video signal 201. The codec system 200 then compresses the segmented video signal 201 into a coded bitstream when functioning as an encoder, as described with respect to steps 105, 107, and 109 in the method 100. When functioning as a decoder, the codec system 200 generates an output video signal from the bitstream, as described with respect to steps 111, 113, 115, and 117 in the operational method 100. Codec system 200 includes an overall coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, an intra-picture prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control analysis component 227, an in-loop filter component 225, a decoded picture buffer component 223, and a header format and context adaptive binary arithmetic coding (CABAC) component 231. Such components are coupled as shown. In FIG. 2, black lines indicate the movement of data to be encoded / decoded, and dashed lines indicate the movement of control data that controls the operation of the other components. The components of codec system 200 may all be present in an encoder. A decoder may include a subset of the components of codec system 200.For example, the decoder may include an intra-picture prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoded picture buffer component 223. These components will now be described.

[0059] The partitioned video signal 201 is a captured video sequence that has been partitioned into blocks of pixels by a coding tree. The coding tree uses various partitioning modes to subdivide the blocks of pixels into smaller blocks of pixels. These blocks can then be further subdivided into smaller blocks. The blocks may be referred to as nodes on the coding tree. Larger parent nodes are partitioned into smaller child nodes. The number of times a node is subdivided is referred to as the depth of the node / coding tree. In some cases, the partitioned blocks may be included in a coding unit (CU). For example, a CU may be a subpart of a CTU that includes a luma block, a red-difference chroma (Cr) block, and a blue-difference chroma (Cb) block, along with corresponding syntax instructions for the CU. Partitioning modes may include a binary tree (BT), a triple tree (TT), and a quad tree (QT), which are used to partition a node into two, three, or four child nodes of varying shapes, respectively, depending on the partitioning mode used. The segmented video signal 201 is forwarded to an overall coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, a filter control analysis component 227 and a motion estimation component 221 for compression.

[0060] The overall coder control component 211 is configured to make decisions related to the coding of images of a video sequence into a bitstream according to application constraints. For example, the overall coder control component 211 manages the optimization of bitrate / bitstream size versus reproduction quality. Such decisions may be made based on storage space / bandwidth availability and image resolution requirements. The overall coder control component 211 also manages buffer utilization taking into account transmission rates to mitigate buffer under-run and over-run issues. To manage these issues, the overall coder control component 211 manages segmentation, prediction, and filtering by other components. For example, the overall coder control component 211 may dynamically increase compression complexity to increase resolution and increase bandwidth usage, or decrease compression complexity to decrease resolution and bandwidth usage. Thus, the overall coder control component 211 controls other components of the codec system 200 to balance bitrate concerns with video signal reproduction quality. The overall coder control component 211 creates control data that controls the operation of other components. Control data is also forwarded to the header format and CABAC component 231 to be encoded into the bitstream for signaling parameters for decoding at the decoder.

[0061] The partitioned video signal 201 is also sent to a motion estimation component 221 and a motion compensation component 219 for inter prediction. A frame or slice of the partitioned video signal 201 may be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform inter predictive coding of the received video blocks relative to one or more blocks in one or more reference frames to provide temporal prediction. The codec system 200 may perform multiple coding passes, for example to select an appropriate coding mode for each block of video data.

[0062] The motion estimation component 221 and the motion compensation component 219 may be highly integrated, but are illustrated separately for conceptual purposes. Motion estimation performed by the motion estimation component 221 is the process of generating motion vectors that estimate motion for a video block. A motion vector may indicate, for example, the displacement of a coded object relative to a predictive block. A predictive block is a block that is found to closely match a block to be coded in terms of pixel differences. A predictive block may also be referred to as a reference block. Such pixel differences may be determined by sum of absolute difference (SAD), sum of square difference (SSD), or other difference metrics. HEVC uses several coded objects including CTUs, coding tree blocks (CTBs), and CUs. For example, a CTU may be divided into CTBs, which may then be divided into CBs for inclusion in a CU. A CU may be coded as a prediction unit (PU) that includes prediction data and / or a transform unit (TU) that includes transform residual data for the CU. The motion estimation component 221 generates the motion vectors, PUs, and TUs by using a rate-distortion analysis as part of a rate-distortion optimization process. For example, the motion estimation component 221 may determine multiple reference blocks, multiple motion vectors, etc. for the current block / frame, and may select the reference block, motion vector, etc. with the best rate-distortion characteristics. The best rate-distortion characteristics balance both the quality of the video reproduction (e.g., the amount of data lost due to compression) and the coding efficiency (e.g., the size of the final encoding).

[0063] In some examples, the codec system 200 may calculate values ​​for sub-integer pixel positions of the reference pictures stored in the decoded picture buffer component 223. For example, the video codec system 200 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of the reference pictures. Thus, the motion estimation component 221 may perform motion search for whole pixel positions and fractional pixel positions and output motion vectors with fractional pixel accuracy. The motion estimation component 221 calculates motion vectors for PUs of video blocks in inter-coded slices by comparing the positions of the PUs with the positions of the predictive blocks of the reference pictures. The motion estimation component 221 outputs the calculated motion vectors as motion data to the header format and CABAC component 231 and motion to the motion compensation component 219 for encoding.

[0064] The motion compensation performed by the motion compensation component 219 may involve fetching or generating a predictive block based on a motion vector determined by the motion estimation component 221. Similarly, in some examples, the motion estimation component 221 and the motion compensation component 219 may be functionally integrated. Upon receiving a motion vector for a PU of a current video block, the motion compensation component 219 may identify a predictive block to which the motion vector points. A residual video block is then formed by subtracting pixel values ​​of the predictive block from pixel values ​​of the current video block being coded to form pixel difference values. Generally, the motion estimation component 221 performs motion estimation on the luma component, and the motion compensation component 219 uses the motion vector calculated based on the luma component for both the chroma and luma components. The predictive block and the residual block are forwarded to the transform scaling and quantization component 213.

[0065] The partitioned video signal 201 is also sent to an intra picture estimation component 215 and an intra picture prediction component 217. As with the motion estimation component 221 and the motion compensation component 219, the intra picture estimation component 215 and the intra picture prediction component 217 may be highly integrated, but are illustrated separately for conceptual purposes. The intra picture estimation component 215 and the intra picture prediction component 217 intra predict the current block relative to blocks in the current frame, as an alternative to the inter prediction performed by the motion estimation component 221 and the motion compensation component 219 between frames, as described above. In particular, the intra picture estimation component 215 determines an intra prediction mode to use to encode the current block. In some examples, the intra picture estimation component 215 selects an appropriate intra prediction mode for encoding the current block from a plurality of tested intra prediction modes. The selected intra prediction mode is then forwarded to the header format and CABAC component 231 for encoding.

[0066] For example, the intra picture estimation component 215 calculates rate-distortion values ​​using a rate-distortion analysis for various tested intra prediction modes and selects the intra prediction mode with the best rate-distortion characteristics among the tested modes. The rate-distortion analysis generally determines the amount of distortion (or error) between the original uncoded block and the coded block that was coded to generate the coded block, and the bit rate (e.g., number of bits) used to generate the coded block. The intra picture estimation component 215 calculates a ratio from the distortion and rate for the various coded blocks to determine which intra prediction mode exhibits the best rate-distortion value for the block. Furthermore, the intra picture estimation component 215 may be configured to code the depth blocks of the depth map using a depth modeling mode (DMM) based on rate-distortion optimization (RDO).

[0067] The intra-picture prediction component 217 may generate a residual block from the prediction block based on a selected intra-picture prediction mode determined by the intra-picture estimation component 215 when implemented in an encoder, or may read the residual block from the bitstream when implemented in a decoder. The residual block includes value differences between the prediction block and the original block, represented as a matrix. The residual block is then forwarded to the transform scaling and quantization component 213. The intra-picture estimation component 215 and the intra-picture prediction component 217 may operate on both the luma and chroma components.

[0068] The transform scaling and quantization component 213 is configured to further compress the residual block. The transform scaling and quantization component 213 applies a transform, such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform, to the residual block to generate a video block including residual transform coefficient values. A wavelet transform, an integer transform, a subband transform, or other types of transforms may also be used. The transform may convert the residual information from a pixel value domain to a transform domain, such as a frequency domain. The transform scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling involves applying a scale factor to the residual information such that different frequency information is quantized with different granularity, which may affect the final visual quality of the reproduced video. The transform scaling and quantization component 213 is also configured to quantize the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, the transform scaling and quantization component 213 may then perform a scan of a matrix containing the quantized transform coefficients. The quantized transform coefficients are forwarded to the header format and CABAC component 231 for encoding into a bitstream.

[0069] The scaling and inverse transform component 229 applies the inverse operations of the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 applies, for example, inverse scaling, transformation and / or quantization to reconstruct the residual block in the pixel domain for later use as a reference block that can become a prediction block for other current blocks. The motion estimation component 221 and / or the motion compensation component 219 may calculate a reference block by adding the residual block back to the corresponding prediction block for use in motion estimation of a later block / frame. Filters are applied to the reconstructed reference block to mitigate artifacts created during the scaling, quantization and transformation. Such artifacts may otherwise cause inaccurate predictions (and create further artifacts) when subsequent blocks are predicted.

[0070] The filter control analysis component 227 and the in-loop filter component 225 apply filters to the residual block and / or the reconstructed image block. For example, the transformed residual block from the scaling and inverse transform component 229 may be combined with the corresponding prediction block from the intra-picture prediction component 217 and / or the motion compensation component 219 to reconstruct the original image block. The filter may then be applied to the reconstructed image block. In some examples, the filter may be applied to the residual block instead. As with the other components in FIG. 2, the filter control analysis component 227 and the in-loop filter component 225 are highly integrated and may be implemented together, but are illustrated separately for conceptual purposes. The filters applied to the reconstructed reference block are applied to a particular spatial region and include multiple parameters to adjust how such filters are applied. The filter control analysis component 227 analyzes the reconstructed reference block and sets the corresponding parameters to determine where such filters should be applied. Such data is forwarded to the header format and CABAC component 231 as filter control data for encoding. The in-loop filter component 225 applies such filters based on the filter control data. The filters may include deblocking filters, noise suppression filters, SAO filters, and adaptive loop filters. Such filters may be applied in the spatial / pixel domain (e.g., reconstructed pixel blocks) or in the frequency domain, depending on the example.

[0071] When operating as an encoder, the reconstructed image blocks, residual blocks and / or predictive blocks after filtering are stored in the decoded picture buffer component 223 for later use in motion estimation as described above. When operating as a decoder, the decoded picture buffer component 223 stores the reconstructed and filtered blocks and forwards them to a display as part of the output video signal. The decoded picture buffer component 223 may be any memory device capable of storing predictive blocks, residual blocks and / or reconstructed image blocks.

[0072] The header format and CABAC component 231 receives data from various components of the codec system 200 and encodes such data into a coded bitstream for transmission towards a decoder. In particular, the header format and CABAC component 231 generates various headers for encoding control data such as global control data and filter control data. In addition, prediction data including intra prediction and motion data, as well as residual data in the form of quantized transform coefficient data, are all encoded into the bitstream. The final bitstream contains all information desired by the decoder to reconstruct the original partitioned video signal 201. Such information may also include an intra prediction mode index table (also called a codeword mapping table), definitions of coding contexts for various blocks, indications of most probable intra prediction modes, indications of partition information, etc. Such data may be encoded by using entropy coding. For example, the information may be encoded using context adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. Following entropy coding, the coded bitstream may be transmitted to another device (e.g., a video decoder) or archived for later transmission or retrieval.

[0073] 3 is a block diagram illustrating an example video encoder 300. The video encoder 300 may be used to implement the encoding functionality of the codec system 200 and / or to implement steps 101, 103, 105, 107 and / or 109 of the method of operation 100. The encoder 300 segments an input video signal, resulting in a segmented video signal 301, which is substantially similar to the segmented video signal 201. The segmented video signal 301 is then compressed and encoded into a bitstream by components of the encoder 300.

[0074] Specifically, the partitioned video signal 301 is forwarded to an intra-picture prediction component 317 for intra prediction. The intra-picture prediction component 317 may be substantially similar to the intra-picture estimation component 215 and the intra-picture prediction component 217. The partitioned video signal 301 is also forwarded to a motion compensation component 321 for inter prediction based on a reference block in a decoded picture buffer component 323. The motion compensation component 321 may be substantially similar to the motion estimation component 221 and the motion compensation component 219. The prediction block and the residual block from the intra-picture prediction component 317 and the motion compensation component 321 are forwarded to a transform and quantization component 313 for transforming and quantizing the residual block. The transform and quantization component 313 may be substantially similar to the transform scaling and quantization component 213. The transformed and quantized residual block and the corresponding prediction block (along with associated control data) are forwarded to an entropy coding component 331 for coding into a bitstream. The entropy coding component 331 may have a header format substantially similar to the CABAC component 231 .

[0075] The transformed and quantized residual block and / or the corresponding prediction block are also forwarded from the transform and quantization component 313 to the inverse transform and quantization component 329 for reconstruction into a reference block for use by the motion compensation component 321. The inverse transform and quantization component 329 may be substantially similar to the scaling and inverse transform component 229. An in-loop filter in the in-loop filter component 325 is also applied to the residual block and / or the reconstructed reference block, depending on the example. The in-loop filter component 325 may be substantially similar to the filter control analysis component 227 and the in-loop filter component 225. The in-loop filter component 325 may include multiple filters as described with respect to the in-loop filter component 225. The filtered block is then stored in the decoded picture buffer component 323 for use as a reference block by the motion compensation component 321. The decoded picture buffer component 323 may be substantially similar to the decoded picture buffer component 223.

[0076] 4 is a block diagram illustrating an exemplary video decoder 400, which may be used to implement the decoding functionality of codec system 200 and / or to implement steps 111, 113, 115 and / or 117 of method of operation 100. Decoder 400 may, for example, receive a bitstream from encoder 300 and generate a reconstructed output video signal based on the bitstream for display to an end user.

[0077] The bitstream is received by the entropy decoding component 433. The entropy decoding component 433 is configured to implement an entropy decoding scheme, such as CAVLC, CABAC, SBAC, PIPE coding, or other entropy coding techniques. For example, the entropy decoding component 433 may use header information to provide a context for interpreting further data encoded as codewords in the bitstream. The decoding information includes any desired information for decoding the video signal, such as global control data, filter control data, partition information, motion data, prediction data, and quantized transform coefficients from the residual block. The quantized transform coefficients are forwarded to the inverse transform and quantization component 429 for reconstruction into the residual block. The inverse transform and quantization component 429 may be similar to the inverse transform and quantization component 329.

[0078] The reconstructed residual block and / or predictive block are forwarded to the intra picture prediction component 417 for reconstruction into an image block based on an intra prediction operation. The intra picture prediction component 417 may be similar to the intra picture estimation component 215 and the intra picture prediction component 217. Specifically, the intra picture prediction component 417 uses a prediction mode to identify a reference block in a frame and applies the residual block to the result to reconstruct an intra predicted image block. The reconstructed intra predicted image block and / or residual block and the corresponding inter prediction data are forwarded to the decoded picture buffer component 423 via the in-loop filter component 425, which may be substantially similar to the decoded picture buffer component 223 and the in-loop filter component 225, respectively. The in-loop filter component 425 filters the reconstructed image block, the residual block and / or the predictive block, and such information is stored in the decoded picture buffer component 423. The reconstructed image blocks from the decoded picture buffer component 423 are forwarded to the motion compensation component 421 for inter prediction. The motion compensation component 421 may be substantially similar to the motion estimation component 221 and / or the motion compensation component 219. Specifically, the motion compensation component 421 uses a motion vector from a reference block to generate a prediction block and applies a residual block to the result to reconstruct an image block. The resulting reconstructed block may also be forwarded to the decoded picture buffer component 423 via the in-loop filter component 425. The decoded picture buffer component 423 continues to store further reconstructed image blocks, which can be reconstructed into frames via the partition information. Such frames may also be arranged into sequences. The sequences are output to a display as a reconstructed output video signal.

[0079] 5A is a schematic diagram illustrating an example picture 500 partitioned into sub-pictures 510. For example, picture 500 may be partitioned for encoding by codec system 200 and / or encoder 300, and may be partitioned for decoding by codec system 200 and / or decoder 400. As another example, picture 500 may be partitioned by the encoder in step 103 of method 100 for use by the decoder in step 111.

[0080] Picture 500 is an image that shows a complete visual portion of a video sequence at a specified time position. Picture 500 may also be referred to as an image and / or a frame. Picture 500 may be designated by a picture order count (POC). POC is an index that indicates the output / display order of picture 500 in a video sequence. Picture 500 may be partitioned into sub-pictures 510. Sub-pictures 510 are rectangular or square regions of one or more slice / tile groups within picture 500. Sub-pictures 510 are optional, thus some video sequences include sub-pictures 510 while others do not. Although four sub-pictures 510 are shown, picture 500 may be partitioned into any number of sub-pictures 510. The partitioning of sub-pictures 510 may be consistent across the entire coded video sequence.

[0081] Subpictures 510 may be used to allow different regions of picture 500 to be treated differently. For example, a given subpicture 510 may be extracted and transmitted to a decoder independently. As a specific example, a user using a virtual reality (VR) headset may see a subset of picture 500, which may provide the user with the impression of being physically present in the space as shown in picture 500. In such a case, streaming only the subpictures 510 that may be displayed to the user may increase coding efficiency. As another example, different subpictures 510 may be treated differently in certain applications. As a specific example, a videoconferencing application may display active speakers at a higher resolution in a more prominent position than users who are not currently speaking. Positioning different users within different subpictures 510 supports real-time reconstruction of the display image to support this function.

[0082] Each subpicture 510 may be identified by a unique subpicture ID, which may be consistent for the entire CVS. For example, the top-left subpicture 510 of picture 500 may have a subpicture ID of 0. In such a case, the top-left subpicture 510 of any picture 500 in the sequence may be referenced by a subpicture ID of 0. Furthermore, each subpicture 510 may include a defined configuration, which may be consistent for the entire CVS. For example, the subpicture 510 may include a height, a width, and / or an offset. The height and width describe the size of the subpicture 510, and the offset describes the position of the subpicture 510. For example, the sum of all the widths of the subpictures 510 in a row is the width of the picture 500. Furthermore, the sum of all the heights of the subpictures 510 in a column is the height of the picture 500. Furthermore, the offset indicates the position of the top-left corner of the subpicture 510 relative to the top-left corner of the picture 500. The height, width and offset of a sub-picture 510 provide sufficient information to position the corresponding sub-picture 510 within the picture 500. Since the partitioning of a sub-picture 510 may be consistent across the entire CVS, parameters related to the sub-picture may be included in a sequence parameter set (SPS).

[0083] 5B is a schematic diagram illustrating an example sub-picture 510 partitioned into slices 515. As shown, the sub-picture 510 of the picture 500 may include one or more slices 515. A slice 515 is an integer number of complete tiles or an integer number of consecutive complete CTU rows within the tiles of the picture that are exclusively contained in a single network abstraction layer (NAL) unit. Although four slices 515 are shown, the sub-picture 510 may include any number of slices 515. The slices 515 include visual data specific to the picture 500 of a given POC. Thus, parameters related to the slices 515 may be included in a picture parameter set (PPS) and / or a slice header.

[0084] 5C is a schematic diagram illustrating an example slice 515 partitioned into tiles 517. As shown, a slice 515 of a picture 500 may include one or more tiles 517. The tiles 517 may be created by partitioning the picture 500 into rectangular rows and columns. Thus, a tile 517 is a rectangular or square region of a CTU within a particular tile column and a particular tile row in a picture. Tiling is optional, and thus some video sequences include tiles 517 while others do not. Although four tiles 517 are shown, a slice 515 may include any number of tiles 517. The tiles 517 may include visual data specific to the slice 515 of the picture 500 of a given POC. In some cases, a slice 515 may also be included in a tile 517. Thus, parameters related to the tiles 517 may be included in the PPS and / or slice header.

[0085] 5D is a schematic diagram illustrating an example slice 515 partitioned into CTUs 519. As shown, a slice 515 (or a tile 517 of a slice 515) of a picture 500 may include one or more CTUs 519. A CTU 519 is a region of a picture 500 that is subdivided by a coding tree to create coding blocks that are encoded / decoded. A CTU 519 may include luma samples for a monochrome picture 500, or a combination of luma and chroma samples for a color picture 500. A grouping of luma or chroma samples that can be partitioned by a coding tree is called a coding tree block (CTB) 518. Thus, a CTU 519 includes a CTB 518 of luma samples and two corresponding CTBs 518 of chroma samples for a picture 500 having a three sample arrangement, or a CTB 518 of samples for a monochrome picture, or a picture coded using a syntax structure used to code three separate color planes and samples.

[0086] As shown above, the picture 500 may be partitioned into sub-pictures 510, slices 515, tiles 517, CTUs 519 and / or CTBs 518, which are then partitioned into blocks. Such blocks are then encoded for transmission to a decoder. Decoding such blocks may result in a decoded image containing various types of noise. To correct such problems, the video coding system may apply various filters across block boundaries. These filters may remove blocking, quantization noise and other undesirable coding artifacts. As noted above, the sub-pictures 510 may be used when performing independent extraction. In this case, the current sub-picture 510 may be decoded and displayed without decoding information from other sub-pictures 510. Thus, block boundaries along the sub-picture 510 edges may align with sub-picture boundaries. In some cases, the block boundaries may also align with tile boundaries. Filters may be applied across such block boundaries, and thus across sub-picture and / or tile boundaries. This may cause errors when the current subpicture 510 is extracted independently, since the filtering process may behave in an unexpected way when data from adjacent subpictures 510 is not available.

[0087] To address these issues, a flag may be used to control filtering at the subpicture 510 level. For example, the flag may be indicated as loop_filter_across_subpic_enabled_flag. When the flag is set for a subpicture 510, the filter can be applied across the corresponding subpicture boundary. When the flag is not set, the filter is not applied across the corresponding subpicture boundary. In this way, the filter can be turned off for subpictures 510 that are coded for separate extraction, or turned on for subpictures 510 that are coded for display as a group. Another flag can be set to control filtering at the tile 517 level. The flag may be indicated as loop_filter_across_tiles_enabled_flag. When the flag is set for a tile 517, the filter can be applied across tile boundaries. When the flag is not set, the filter is not applied across tile boundaries. In this way, the filter can be turned off or on for use at tile boundaries (e.g., while continuing to filter the interior of the tile). As used herein, a filter is applied across a subpicture 510 or tile 517 boundary when the filter is applied to samples on both sides of the boundary.

[0088] Also, as noted above, tiling is optional. However, some video coding systems describe subpicture boundaries in terms of tiles 517 contained in the subpicture 510. In such systems, the subpicture boundary description in terms of tiles 517 limits the use of the subpicture 510 to pictures 500 that use tiles 517. To expand the applicability of the subpicture 510, the subpicture 510 may be described in terms of boundaries in terms of CTBs 518 and / or CTUs 519. Specifically, the width and height of the subpicture 510 may be signaled in units of CTBs 518. Furthermore, the position of the top-left CTU 519 of the subpicture 510 may be signaled as an offset from the top-left CTU 519 of the picture 500 as measured in CTBs 518. The CTU 519 and CTB 518 sizes may be set to predefined values. Thus, signaling the sub-picture dimensions and position with respect to CTB 518 and CTU 519 provides sufficient information for a decoder to position sub-picture 510 for display. This allows sub-picture 510 to be used even when tiles 517 are not used.

[0089] Furthermore, some video coding systems address slices 515 based on their location relative to picture 500. This creates a problem when subpictures 510 are coded for independent extraction and display. In such cases, slices 515 and corresponding addresses associated with omitted subpictures 510 are also omitted. Omission of the address of slice 515 may prevent a decoder from properly locating slice 515. Some video coding systems address this problem by dynamically rewriting the address in the slice header associated with slice 515. Since a user may request any subpicture, such rewriting occurs every time a user requests a video, which is very resource intensive. To overcome this problem, slices 515 are addressed relative to the subpicture 510 that contains slice 515 when subpicture 510 is used. For example, slice 515 can be identified by an index or other value that is specific to the subpicture 510 that contains slice 515. A slice address can be coded in a slice header associated with slice 515. The sub-picture ID of the sub-picture 510 containing the slice 515 can also be coded into the slice header. Furthermore, the dimensions / configuration of the sub-picture 510 can be coded into the SPS together with the sub-picture ID. Thus, the decoder can obtain the sub-picture 510 configuration from the SPS based on the sub-picture ID and position the slice 515 in the sub-picture 510 without referring to the complete picture 500. Thus, rewriting of the slice header can be omitted when the sub-picture 510 is extracted, which significantly reduces resource usage in the encoder, decoder and / or corresponding slicer.

[0090] Once the picture 500 is partitioned into CTBs 518 and / or CTUs 519, the CTBs 518 and / or CTUs 519 can be further divided into coding blocks. The coding blocks can then be coded according to intra-prediction and / or inter-prediction. This disclosure also includes improvements related to inter-prediction mechanisms. Inter-prediction can be performed in several different modes that can operate according to unidirectional inter-prediction and / or bidirectional inter-prediction.

[0091] 6 is a schematic diagram illustrating an example of unidirectional inter prediction 600 that may be performed to determine motion vectors (MVs), for example, in block compression step 105, block decoding step 113, motion estimation component 221, motion compensation component 219, motion compensation component 321 and / or motion compensation component 421. For example, unidirectional inter prediction 600 may be used to determine motion vectors for encoded and / or decoded blocks created when partitioning a picture such as picture 500.

[0092] The unidirectional inter prediction 600 uses a reference frame 630 having a reference block 631 to predict a current block 611 in a current frame 610. The reference frame 630 may be temporally located after the current frame 610 as shown (e.g., as a subsequent reference frame), but in some examples may be temporally located before the current frame 610 (e.g., as a preceding reference frame). The current frame 610 is an exemplary frame / picture that is encoded / decoded at a particular time. The current frame 610 includes an object in a current block 611 that matches an object in a reference block 631 of the reference frame 630. The reference frame 630 is a frame used as a reference for encoding the current frame 610, and the reference block 631 is a block in the reference frame 630 that includes an object that is also included in the current block 611 of the current frame 610.

[0093] The current block 611 is any coding unit being coded / decoded at a specified point in the coding process. The current block 611 may be an entire partitioned block or a sub-block when using affine inter prediction mode. The current frame 610 is separated from the reference frame 630 by some temporal distance (TD) 633. The TD 633 indicates the amount of time between the current frame 610 and the reference frame 630 in the video sequence and may be measured in units of frames. The prediction information for the current block 611 may reference the reference frame 630 and / or the reference block 631 by a reference index indicating the direction and the temporal distance between the frames. Over the period represented by the TD 633, an object in the current block 611 moves from a position in the current frame 610 to another position in the reference frame 630 (e.g., the position of the reference block 631). For example, the object may move along a motion trajectory 613, which is the direction of the object's movement over time. The motion vector 635 describes the direction and magnitude of the object's motion along the motion trajectory 613 across the TD 633. Thus, the coded motion vector 635, the reference block 631, and the residual, which comprises the difference between the current block 611 and the reference block 631, provide sufficient information to reconstruct the current block 611 and position it within the current frame 610.

[0094] 7 is a schematic diagram illustrating an example of bidirectional inter prediction 700 that may be performed to determine MVs, for example, in block compression step 105, block decoding step 113, motion estimation component 221, motion compensation component 219, motion compensation component 321 and / or motion compensation component 421. For example, bidirectional inter prediction 700 may be used to determine motion vectors for encoded and / or decoded blocks created when partitioning a picture, such as picture 500.

[0095] Bidirectional inter prediction 700 is similar to unidirectional inter prediction 600, but uses a pair of reference frames to predict a current block 711 in a current frame 710. Thus, the current frame 710 and the current block 711 are substantially similar to the current frame 610 and the current block 611, respectively. The current frame 710 is temporally positioned between a previous reference frame 720, which occurs before the current frame 710 in the video sequence, and a subsequent reference frame 730, which occurs after the current frame 710 in the video sequence. The previous reference frame 720 and the subsequent reference frame 730 are otherwise substantially similar to reference frame 630.

[0096] A current block 711 is matched with a previous reference block 721 in a previous reference frame 720 and with a subsequent reference block 731 in a subsequent reference frame 730. Such matching indicates that an object moves from a position in the previous reference block 721 along a motion trajectory 713 through the current block 711 to a position in the subsequent reference block 731 during the course of the video sequence. The current frame 710 is separated from the previous reference frame 720 by some previous time distance (TD0) 723 and from the subsequent reference frame 730 by some subsequent time distance (TD1) 733. TD0 723 indicates the amount of time, in units of frames, between the previous reference frame 720 and the current frame 710 in the video sequence. TD1 733 indicates the amount of time, in units of frames, between the current frame 710 and the subsequent reference frame 730 in the video sequence. Thus, the object moves along the motion trajectory 713 from the previous reference block 721 to the current block 711 over the period indicated by TD0 723. The object also moves along the motion trajectory 713 from the current block 711 to the subsequent reference block 731 over the period indicated by TD1 733. Prediction information for the current block 711 may refer to the previous reference frame 720 and / or previous reference block 721, and the subsequent reference frame 730 and / or subsequent reference block 731, by a pair of reference indexes indicating the direction and time distance between the frames.

[0097] A previous motion vector (MV0) 725 describes the direction and magnitude of an object's motion along a motion trajectory 713 (e.g., between a previous reference frame 720 and a current frame 710) across TD0 723. A subsequent motion vector (MV1) 735 describes the direction and magnitude of an object's motion along a motion trajectory 713 (e.g., between a current frame 710 and a subsequent reference frame 730) across TD1 733. Thus, in bidirectional inter prediction 700, a current block 711 can be coded and reconstructed by using a previous reference block 721 and / or a subsequent reference block 731, MV0 725, and MV1 735.

[0098] In both merge and advanced motion vector prediction (AMVP) modes, the candidate list is generated by adding candidate motion vectors to the candidate list in an order defined by a candidate list determination pattern. Such candidate motion vectors may include motion vectors according to unidirectional inter prediction 600, bidirectional inter prediction 700, or a combination thereof. Specifically, motion vectors are generated for neighboring blocks when such blocks are coded. Such motion vectors are added to a candidate list for a current block, and a motion vector for the current block is selected from the candidate list. The motion vector can then be signaled as an index of the selected motion vector in the candidate list. The decoder can build the candidate list using the same process as the encoder and determine the selected motion vector from the candidate list based on the signaled index. Thus, the candidate motion vectors include motion vectors generated according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700, depending on which approach is used when such neighboring blocks are coded.

[0099] 8 is a schematic diagram illustrating an example 800 of coding a current block 801 based on candidate motion vectors from a neighboring coded block 802. Using the method of operation 100 of the encoder 300 and / or the decoder 400 and / or the functionality of the codec system 200, the neighboring blocks 802 can be used to generate a candidate list. Such a candidate list can be used in inter prediction by unidirectional inter prediction 600 and / or bidirectional inter prediction 700. The candidate list can then be used to encode / decode the current block 801, which may be generated by partitioning a picture such as the picture 500.

[0100] The current block 801 is a block being coded at an encoder or decoded at a decoder at a specified time, depending on the example. The coded block 802 is a block that has already been coded at a specified time. Thus, the coded block 802 is potentially available for use when generating the candidate list. The current block 801 and the coded block 802 may be included in a common frame and / or may be included in a temporally adjacent frame. When the coded block 802 is included in a common frame with the current block 801, the coded block 802 includes a boundary that is immediately adjacent (e.g., touching) the boundary of the current block 801. When the coded block 802 is included in a temporally adjacent frame, the coded block 802 is located in the same position in the temporally adjacent frame as the position of the current block 801 in the current frame. The candidate list can be generated by adding the motion vector from the coded block 802 as a candidate motion vector. The current block 801 can then be coded by selecting a candidate motion vector from the candidate list and signaling an index of the selected candidate motion vector.

[0101] 9 is a schematic diagram illustrating an example pattern 900 for determining a candidate list of motion vectors. In particular, the operating method 100 of the encoder 300 and / or the decoder 400 and / or the use of the functionality of the codec system 200 can use the candidate list determination pattern 900 to use in generating a candidate list 911 for encoding a current block 801 partitioned from a picture 500. The resulting candidate list 911 can be a merge candidate list or an AMVP candidate list, which can be used in inter prediction with unidirectional inter prediction 600 and / or bidirectional inter prediction 700.

[0102] When encoding a current block 901, the candidate list determination pattern 900 searches locations 905, denoted as A0, A1, B0, B1 and / or B2, in the same picture / frame as the current block for a valid candidate motion vector. The candidate list determination pattern 900 may also search co-located blocks 909 for a valid candidate motion vector. A co-located block 909 is a block 901 that is co-located with the current block 901, but is included in a temporally adjacent picture / frame. The candidate motion vectors may then be arranged in a candidate list 911 in a predefined examination order. The candidate list 911 is thus a procedurally generated list of indexed candidate motion vectors.

[0103] The candidate list 911 can be used to select a motion vector for performing inter prediction for the current block 901. For example, the encoder can obtain samples of the reference block pointed to by the candidate motion vector from the candidate list 911. The encoder can then select the candidate motion vector that points to the reference block that most closely matches the current block 901. The index of the selected candidate motion vector can then be encoded to represent the current block 901. In some cases, the candidate motion vector points to a reference block that includes partial reference samples 915. In this case, the interpolation filter 913 can be used to reconstruct the complete reference samples 915 to support the motion vector selection. The interpolation filter 913 is a filter that can upsample a signal. Specifically, the interpolation filter 913 is a filter that can accept a partial / lower quality signal as input and determine an approximation of a more complete / higher quality signal. Thus, the interpolation filter 913 can be used in certain cases to obtain a complete set of reference samples 915 for use in selecting a reference block for the current block 901 and thus in selecting a motion vector for encoding the current block 901.

[0104] The above mechanism for coding a block based on inter prediction by using a candidate list may cause certain errors when a sub-picture such as sub-picture 510 is used. In particular, problems may arise when the current block 901 is included in the current sub-picture, but the motion vector points to a reference block that is at least partially located in a neighboring sub-picture. In such a case, the current sub-picture may be extracted for presentation without the neighboring sub-picture. When this occurs, a portion of the reference block in the neighboring sub-picture may not be transmitted to the decoder, and thus the reference block may not be available for decoding the current block 901. When this occurs, the decoder does not have access to sufficient data to decode the current block 901.

[0105] This disclosure provides a mechanism to address this problem. In one example, a flag is used to indicate that the current subpicture should be treated as a picture. This flag can be set to support separate extraction of subpictures. Specifically, when the flag is set, the current subpicture should be coded without reference to data in other subpictures. In this case, the current subpicture is treated like a picture in that it is coded separately from other subpictures and can be displayed as a separate picture. Thus, this flag may be indicated as subpic_treated_as_pic_flag[i], where i is the index of the current subpicture. When the flag is set, the motion vector candidates (also known as motion vector predictors) obtained from the co-located block 909 include only motion vectors that point into the current subpicture. Any motion vector predictors that point outside the current subpicture are excluded from the candidate list 911. This ensures that motion vectors that point outside the current subpicture are not selected and the associated errors are avoided. This example applies specifically to motion vectors from the co-located block 909. Motion vectors from search positions 905 within the same picture / frame may be corrected by different mechanisms, as described below.

[0106] Another example may be used to address the search location 905 when the current subpicture is treated as a picture (e.g., when subpic_treated_as_pic_flag[i] is set). When the current subpicture is treated like a picture, the current subpicture should be extracted without reference to other subpictures. An exemplary mechanism relates to an interpolation filter 913. An interpolation filter 913 can be applied to a sample at one location to interpolate (e.g., predict) a related sample at another location. In this example, a motion vector from a coded block at the search location 905 may point to such a reference sample 915 outside the current subpicture as long as the interpolation filter 913 can interpolate such a reference sample 915 based only on the reference sample 915 from the current subpicture. Thus, this example uses a clipping function that is applied when applying the interpolation filter 913 to a motion vector candidate from the search location 905 from the same picture. This clipping function clips data from adjacent sub-pictures, thus removing such data as input to the interpolation filter 913 when determining reference samples 915 pointed to by motion vector candidates. This approach maintains separation between sub-pictures during encoding to support separate extraction and decoding when the sub-pictures are treated as pictures. The clipping function may be applied to the luma sample bilinear interpolation process, the luma sample 8-tap interpolation filtering process, and / or the chroma sample interpolation process.

[0107] 10 is a block diagram illustrating an exemplary in-loop filter 1000. The in-loop filter 1000 may be used to realize the in-loop filters 225, 325, and / or 425. Furthermore, the in-loop filter 1000 may be applied at the encoder and the decoder when performing the method 100. Furthermore, the in-loop filter 1000 may be applied to filter a current block 801 partitioned from the picture 500, which may be coded according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 based on a candidate list generated according to the pattern 900. The in-loop filter 1000 includes a deblocking filter 1043, an SAO filter 1045, and an adaptive loop filter (ALF) 1047. The filters of the in-loop filter 1000 are sequentially applied to a reconstructed image block at the encoder (e.g., before use as a reference block) and at the decoder before display.

[0108] The deblocking filter 1043 is configured to remove block-shaped edges created by block-based inter- and intra-prediction. The deblocking filter 1043 scans an image portion (e.g., an image slice) for discontinuities in chroma and / or luma values ​​that occur at partition boundaries. The deblocking filter 1043 then applies a smoothing function to the block boundaries to remove such discontinuities. The strength of the deblocking filter 1043 may be modified depending on the spatial activity (e.g., variance of luma / chroma components) occurring in the areas adjacent to the block boundaries.

[0109] The SAO filter 1045 is configured to remove artifacts related to sample distortion caused by the encoding process. The SAO filter 1045 in the encoder classifies the deblocked samples of the reconstructed image into several categories based on the relative deblocking edge shape and / or direction. An offset is then determined and added to the samples based on the category. The offset is then coded into the bitstream and used by the SAO filter 1045 in the decoder. The SAO filter 1045 removes banding artefacts (bands of values ​​rather than smooth transitions) and ringing artefacts (spurious signals near sharp edges).

[0110] The ALF 1047 is configured at the encoder to compare the reconstructed image with the original image. The ALF 1047 determines coefficients that describe the differences between the reconstructed image and the original image, for example via a Wiener-based adaptive filter. Such coefficients are coded into the bitstream and used by the ALF 1047 at the decoder to remove the differences between the reconstructed image and the original image.

[0111] Image data filtered by in-loop filter 1000 is output to a picture buffer 1023, which is substantially similar to decoded picture buffer 223, 323 and / or 423. As noted above, the deblocking filter 1043, SAO filter 1045 and / or ALF 1047 may be turned off at sub-picture and / or tile boundaries via flags such as loop_filter_across_subpic_enabled flag and / or loop_filter_across_tiles_enabled_flag, respectively.

[0112] 11 is a schematic diagram illustrating an example bitstream 1100 including coding tool parameters for supporting decoding sub-pictures of a picture. For example, the bitstream 1100 can be generated by the codec system 200 and / or the encoder 300 for decoding by the codec system 200 and / or the decoder 400. As another example, the bitstream 1100 may be generated by the encoder in step 109 of the method 100 for use by the decoder in step 111. Furthermore, the bitstream 1100 may include the encoded picture 500, the corresponding sub-picture 510, and / or associated coded blocks such as the current block 801 and / or 901, which may be coded according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 based on the candidate list generated according to the pattern 900. The bitstream 1100 may also include parameters for configuring the in-loop filter 1000.

[0113] The bitstream 1100 includes a sequence parameter set (SPS) 1110, a number of picture parameter sets (PPS) 1111, a number of slice headers 1115, and image data 1120. The SPS 1110 includes sequence data common to all pictures in a video sequence included in the bitstream 1100. Such data may include picture size, bit depth, coding tool parameters, bit rate limits, etc. The PPS 1111 includes parameters that apply to the entire picture. Thus, each picture in the video sequence may reference the PPS 1111. It should be noted that while each picture references the PPS 1111, in some examples, a single PPS 1111 may include data for multiple pictures. For example, multiple similar pictures may be coded according to similar parameters. In such a case, a single PPS 1111 may include data for such similar pictures. The PPS 1111 may indicate coding tools, quantization parameters, offsets, etc. available for slices in the corresponding picture. The slice header 1115 contains parameters specific to each slice in a picture. Thus, there may be one slice header 1115 per slice in a video sequence. The slice header 1115 may include slice type information, picture order count (POC), reference picture list, prediction weights, tile entry points, deblocking parameters, etc. It should be noted that the slice header 1115 may also be referred to as a tile group header in some circumstances.

[0114] The image data 1120 includes video data coded according to inter-prediction and / or intra-prediction and corresponding transformed and quantized residual data. For example, a video sequence includes multiple pictures coded as image data. A picture is a single frame of a video sequence and is therefore typically displayed as a single unit when displaying a video sequence. However, sub-pictures may be displayed to realize certain technologies such as virtual reality, picture-in-picture, etc. The pictures each refer to a PPS 1111. A picture is divided into sub-pictures, tiles, and / or slices as described above. In some systems, a slice is referred to as a tile group that contains tiles. The tile group of slices and / or tiles refers to a slice header 1115. The slices are further divided into CTUs and / or CTBs. The CTUs / CTBs are further divided into coding blocks based on a coding tree. The coding blocks can then be coded / decoded according to a prediction mechanism.

[0115] The parameter set in the bitstream 1100 includes various data that can be used to implement the examples described herein. To support the implementation of the first example, the SPS 1110 of the bitstream 1100 includes a sub-picture treated as picture flag 1131 for a specified sub-picture. In some examples, the sub-picture treated as picture flag 1131 is denoted as subpic_treated_as_pic_flag[i], where i is the index of the sub-picture associated with the flag. For example, the sub-picture treated as picture flag 1131 may be set equal to 1 to specify that the i-th sub-picture of the respective coded picture in the coded video sequence (in the image data 1120) is treated as a picture in the decoding process except for in-loop filtering operations. The sub-picture treated as picture flag 1131 may be used when the current sub-picture in the current picture is coded according to inter prediction. When the sub-picture treated as a picture flag 1131 is set to indicate that the current sub-picture is treated as a picture, a candidate list of candidate motion vectors for the current block can be determined by excluding from the candidate list co-located motion vectors that are contained in the co-located block and point outside the current sub-picture, which ensures that when the current sub-picture is extracted separately from other sub-pictures, motion vectors that point outside the current sub-picture are not selected and associated errors are avoided.

[0116] In some examples, the candidate list of motion vectors for the current block is determined according to temporal luma motion vector prediction. For example, when the current block is a luma block of luma samples, the selected current motion vector for the current block is a temporal luma motion vector that points to a reference luma sample in a reference block, and the current block is coded based on the reference luma sample, temporal luma motion vector prediction may be used. In such a case, the temporal luma motion vector prediction is performed according to the following: xColBr = xCb + cbWidth; yColBr=yCb+cbHeight; rightBoundaryPos=subpic_treated_as_pic_flag[SubPicIdx]? SubPicRightBoundaryPos:pic_width_in_luma_samples-1; and botBoundaryPos=subpic_treated_as_pic_flag[SubPicIdx]? SubPicBotBoundaryPos:pic_height_in_luma_samples-1 where xColBr and yColBR specify the position of the co-located block, xCb and yCb specify the top-left sample of the current block relative to the top-left sample of the current picture, cbWidth is the width of the current block, cbHeight is the height of the current block, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and pic_width_in_luma_samples is the width of the current picture measured in luma samples. , pic_height_in_luma_samples is the height of the current picture measured in luma samples, botBoundaryPos is the calculated position of the bottom boundary of the subpicture, rightBoundaryPos is the calculated position of the right boundary of the subpicture, SubPicIdx is the index of the subpicture, co-located motion vectors are excluded when yCb>>CtbLog2SizeY is not equal to yColBr>>CtbLog2SizeY, and CtbLog2SizeY indicates the size of the coding tree block.

[0117] The sub-picture treated as picture flag 1131 may also be used for the implementation of the second example. As in the first example, the sub-picture treated as picture flag 1131 may be used when the current sub-picture in the current picture is coded according to inter prediction. In this example, a motion vector can be determined (e.g., from a candidate list) for the current block of the sub-picture. When the sub-picture treated as picture flag 1131 is set, a clipping function can be applied to a sample location in the reference block. The sample location is a location in the picture that can contain a single sample including a luma value and / or a pair of chroma values. An interpolation filter can then be applied when the motion vector points outside the current sub-picture. This clipping function ensures that the interpolation filter does not rely on data from adjacent sub-pictures to maintain separation between sub-pictures to support separate extraction.

[0118] The clipping function may be applied in the luma sample bilinear interpolation process. The luma sample bilinear interpolation process may receive an input including a luma position in whole sample units (xIntL, yIntL). The luma sample bilinear interpolation process outputs a predicted luma sample value (predSampleLXL). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following holds: xInti = Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xIntL + i), and yInti=Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yIntL+i) where subpic_treated_as_pic_flag is a flag that is set to indicate the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and Clip3 is a clipping function according to

number

[0119] The clipping function can also be applied in the luma sample 8-tap interpolation filtering process. The luma sample 8-tap interpolation filtering process receives an input including a luma position in whole sample units (xIntL, yIntL). The luma sample bilinear interpolation process outputs a predicted luma sample value (predSampleLXL). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following applies: xInti = Clip3 (SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xIntL + i - 3), and yInti=Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yIntL+i-3) where subpic_treated_as_pic_flag is a flag that is set to indicate the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and Clip3 is as above.

[0120] The clipping function can also be applied in the chroma sample interpolation process. The chroma sample interpolation process receives an input containing a chroma position in whole sample units (xIntC, yIntC). The chroma sample interpolation process outputs a predicted chroma sample value (predSampleLXC). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following holds: xInti = Clip3 (SubPicLeftBoundaryPos / SubWidthC, SubPicRightBoundaryPos / SubWidthC, xIntC+i), and yInti=Clip3(SubPicTopBoundaryPos / SubHeightC,SubPicBotBoundaryPos / SubHeightC,yIntC+i) where subpic_treated_as_pic_flag is a flag that is set to indicate that the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, SubWidthC and SubHeightC indicate the horizontal and vertical sampling rate ratios between the luma and chroma samples, and Clip3 is as above.

[0121] The loop filter enable flag 1132 across subpictures in the SPS 1110 may be used for the implementation of the third example. The loop filter enable flag 1132 across subpictures may be set to control whether filtering is used across a specified subpicture boundary. For example, the loop filter enable flag 1132 across subpictures may be indicated as loop_filter_across_subpic_enabled_flag. The loop filter enable flag 1132 across subpictures may be set to 1 when specifying that an in-loop filtering operation can be performed across a subpicture boundary, or may be set to 0 when specifying that an in-loop filtering operation is not performed across a subpicture boundary. Thus, a filtering operation may or may not be performed across a subpicture boundary based on the value of the loop filter enable flag 1132 across subpictures. The filtering operations may include applying a deblocking filter 1043, an ALF 1047, and / or an SAO filter 1045. Thus, the filters can be turned off for subpictures that are coded for separate extraction, or turned on for subpictures that are coded for display as a group.

[0122] A loop filter enable flag 1134 across tiles in the PPS 1111 may be used for the implementation of the fourth example. The loop filter enable flag 1134 across tiles may be set to control whether filtering is used across a specified tile boundary. For example, the loop filter enable flag 1134 across tiles may be indicated as loop_filter_across_tiles_enabled_flag. The loop filter enable flag 1134 across tiles may be set to 1 when specifying that an in-loop filtering operation can be performed across a tile boundary, or may be set to 0 when specifying that an in-loop filtering operation is not performed across a tile boundary. Thus, a filtering operation may or may not be performed across a specified tile boundary based on the value of the loop filter enable flag 1134 across tiles. The filtering operation may include applying a deblocking filter 1043, an ALF 1047, and / or an SAO filter 1045.

[0123] Subpicture data 1133 in SPS 1110 may be used for the implementation of the fifth example. Subpicture data 1133 may include a width, height, and offset for each subpicture in image data 1120. For example, the width and height of each subpicture may be described in subpicture data 1133 in units of CTB. In some examples, the width and height of a subpicture may be stored in subpicture data 1133 as subpic_width_minus1 and subpic_height_minus1, respectively. Furthermore, the offset of each subpicture may be described in subpicture data 1133 in units of CTU. For example, the offset of each subpicture may be specified as the vertical position and horizontal position of the top-left CTU of the subpicture. Specifically, the offset of a subpicture may be specified as the difference between the top-left CTU of the picture and the top-left CTU of the subpicture. In some examples, the vertical and horizontal positions of the top-left CTU of a subpicture are stored as subpic_ctu_top_left_y and subpic_ctu_top_left_x, respectively, in subpicture data 1133. This example implementation describes subpictures in subpicture data 1133 in terms of CTBs / CTUs rather than in terms of tiles. This allows subpictures to be used even when tiles are not used in the corresponding picture / subpicture.

[0124] The subpicture data 1133 in the SPS 1110, the slice address 1136 in the slice header 1115, and the slice subpicture ID 1135 in the slice header 1115 may be used for the implementation of the sixth example. The subpicture data 1133 may be implemented as described in the implementation of the fifth example. The slice address 1136 may include a subpicture-level slice index of the slice (e.g., image data 1120) associated with the slice header 1115. For example, slices are indexed based on the location of the slice in the subpicture, rather than based on the location of the slice in the picture. The slice address 1136 may be stored in a slice_address variable. The slice subpicture ID 1135 includes an ID of the subpicture that includes the slice associated with the slice header 1115. Specifically, the slice subpicture ID 1135 may reference a description (e.g., width, height, and offset) of the corresponding subpicture in the subpicture data 1133. The slice subpicture ID 1135 may be stored in the slice_subpic_id variable. Thus, the slice address 1136 is signaled as an index based on the position of the slice within the subpicture indicated by the subpicture ID 1135 as described in the subpicture data 1133. In this way, the position of the slice within the subpicture can be determined even when the subpictures are extracted separately and other subpictures are omitted from the bitstream 1100. This is because this addressing scheme separates the address of each subpicture from the other subpictures. Thus, the slice header 1115 does not need to be rewritten when a subpicture is extracted, as would be required in an addressing scheme in which slices are addressed based on the position of the slice within the picture. It should be noted that this approach may be used when the slice is a rectangular slice (as opposed to a raster scan slice). For example, the rect_slice_flag in the PPS 1111 may be set equal to 1 to indicate that the slice is a rectangular slice.

[0125] An exemplary implementation of sub-pictures used in some video coding systems is as follows: Information related to sub-pictures that may be present in the CVS may be signaled in the SPS. Such signaling may include the following information: The number of sub-pictures present in each picture of the CVS may be included in the SPS. In the context of the SPS or CVS, the co-located sub-pictures for all access units (AUs) may be collectively referred to as a sub-picture sequence. A loop for further specifying information related to the characteristics of each sub-picture may also be included in the SPS. Such information may include the sub-picture identification, the sub-picture's location (e.g., the offset distance between the sub-picture's top-left corner luma sample and the picture's top-left corner luma sample), and the sub-picture's size. Furthermore, the SPS may also be used to signal whether each of the sub-pictures is a motion constrained sub-picture, a motion constrained sub-picture being a sub-picture that includes an MCTS. Profile, tier and level information for each sub-picture may be included in the bitstream, unless such information is otherwise derivable. Such information may be used for profile, tier and level information for the extracted bitstream created from extracting the sub-pictures from the original bitstream containing the entire picture. The profile and tier of each sub-picture may be derived to be the same as the original profile and tier. The level for each sub-picture may be explicitly signaled. Such signaling may be present in the loop above. Sequence-level hypothetical reference decoder (HRD) parameters may be signaled in the video usability information (VUI) part of the SPS for each sub-picture (or equivalently, each sub-picture sequence).

[0126] When a picture is not partitioned into two or more sub-pictures, the characteristics of the sub-pictures (e.g., position, size, etc.) may not be signaled in the bitstream, except for the sub-picture ID. When a sub-picture in a picture in the CVS is extracted, each access unit in the new bitstream may not contain a sub-picture. This is because the resulting image data in each AU in the new bitstream is not partitioned into multiple sub-pictures. Therefore, sub-picture characteristics such as position and size may be omitted from the SPS, since such information can be derived from the picture characteristics. However, the sub-picture identification is still signaled, since this ID may be referenced by the video coding layer (VCL) NAL units / tile groups included in the extracted sub-picture. To reduce resource usage, changing the sub-picture ID should be avoided when extracting a sub-picture.

[0127] The position (x offset and y offset) of the subpicture within a picture may be signaled in units of luma samples and may represent the distance between the top-left corner luma sample of the subpicture and the top-left corner luma sample of the picture. In another example, the position of the subpicture within a picture may be signaled in units of the minimum coding luma block size (MinCbSizeY) and may represent the distance between the top-left corner luma sample of the subpicture and the top-left corner luma sample of the picture. In another example, the unit of the subpicture position offset may be explicitly indicated by a syntax element in the parameter set, and the unit may be CtbSizeY, MinCbSizeY, luma samples, or other values. The codec may require the width of the subpicture to be an integer multiple of the luma CTU size (CtbSizeY) when the right border of the subpicture does not coincide with the right border of the picture. Similarly, the codec may further require that the subpicture height be an integer multiple of CtbSizeY when the bottom boundary of the subpicture does not coincide with the bottom boundary of the picture. The codec may also require that the subpicture be located at the rightmost position of the picture when the subpicture width is not an integer multiple of the luma CTU size. Similarly, the codec may require that the subpicture be located at the bottommost position of the picture when the subpicture height is not an integer multiple of the luma CTU size. When the subpicture width is signaled in units of the luma CTU size and the subpicture width is not an integer multiple of the luma CTU size, the actual width in luma samples may be derived based on the offset position of the subpicture, the subpicture width in luma CTU size, and the picture width in luma samples. Similarly, when the subpicture height is signaled in units of luma CTU size and the subpicture height is not an integer multiple of the luma CTU size, the actual height in luma samples can be derived based on the offset position of the subpicture, the subpicture height in luma CTU size, and the picture height in luma samples.

[0128] For any subpicture, the subpicture ID may be different from the subpicture index. The subpicture index may be the index of the subpicture as signaled in the loop of subpictures in the SPS. Alternatively, the subpicture index may be an index assigned in the subpicture raster scan order for the picture. When the value of the subpicture ID of each subpicture is the same as its subpicture index, the subpicture ID may be signaled or derived. When the subpicture ID of each subpicture is different from its subpicture index, the subpicture ID is explicitly signaled. The number of bits for signaling the subpicture ID may be signaled in the same parameter set (e.g., in the SPS) that contains the subpicture characteristics. Some values ​​for the subpicture ID may be reserved for specific purposes. Such value reservation may be as follows: When a tile group / slice header includes a sub-picture ID to specify which sub-pictures include a tile group, the value 0 may be reserved and not used for sub-pictures to ensure that the first few bits of the tile group / slice header are not all 0 and to avoid generating emulation prevention code. When the sub-pictures of a picture do not cover the entire area of ​​the picture without overlaps and gaps, a value (e.g., value 1) may be reserved for tile groups that are not part of any sub-picture. Alternatively, the sub-picture IDs of the remaining areas may be explicitly signaled. The number of bits for signaling the sub-picture ID may be constrained as follows: The range of values ​​should be sufficient to uniquely identify all sub-pictures in the picture, including the reserved values ​​of the sub-picture ID. For example, the minimum number of bits for the sub-picture ID may be the value of Ceil(Log2(number of sub-pictures in the picture + number of reserved sub-picture IDs).

[0129] The collection of sub-pictures in a loop may be required to cover the entire picture without gaps and without overlaps. When this constraint is applied, there is a flag for each sub-picture to specify whether the sub-picture is a motion constrained sub-picture, meaning that the sub-picture can be extracted. Alternatively, the collection of sub-pictures may not cover the entire picture. However, there may be no overlap between the sub-pictures of a picture.

[0130] The sub-picture ID may be present immediately after the NAL unit header so that the extractor does not need to understand the rest of the NAL unit bits to assist the sub-picture extraction process. For VCL NAL units, the sub-picture ID may be present in the first bit of the tile group header. For non-VCL NAL units, the following conditions may apply: The sub-picture ID may not need to be present immediately after the NAL unit header for an SPS. For a PPS, when all tile groups of the same picture are constrained to refer to the same PPS, the sub-picture ID does not need to be present immediately after the NAL unit header. On the other hand, if tile groups of the same picture are allowed to refer to different PPSs, the sub-picture ID may be present in the first bit of the PPS (e.g., immediately after the PPS NAL unit header). In this case, two different tile groups of one picture are not allowed to share the same PPS. Alternatively, when tile groups of the same picture are allowed to refer to different PPSs and different tile groups of the same picture are also allowed to share the same PPS, the sub-picture ID is not present in the PPS syntax. Alternatively, when tile groups of the same picture are allowed to reference different PPSs and different tile groups of the same picture are allowed to share the same PPS, a list of sub-picture IDs is present in the PPS syntax. The list indicates the sub-pictures to which the PPS applies. For other non-VCL NAL units, if the non-VCL unit applies at or above the picture level (e.g., access unit delimiter, end of sequence, end of bitstream, etc.), the sub-picture ID does not need to be present immediately after its NAL unit header. Otherwise, the sub-picture ID may be present immediately after the NAL unit header.

[0131] Tile partitioning within individual subpictures may be signaled in the PPS, but tile groups within the same picture are allowed to reference different PPSs. In this case, tiles are grouped within each subpicture, not across pictures. Thus, the concept of tile grouping in such cases includes partitioning the subpictures into tiles. Alternatively, a Sub-Picture Parameter Set (SPPS) may be used to describe the tile partitioning within individual subpictures. The SPPS references the SPS by using a syntax element that references the SPS ID. The SPPS may include the subpicture ID. For the purpose of subpicture extraction, the syntax element that references the subpicture ID is the first syntax element in the SPPS. The SPPS includes a tile structure indicating the number of columns, number of rows, uniform tile spacing, etc. The SPPS may include a flag to indicate whether the loop filter is valid across the associated subpicture boundary. Alternatively, the subpicture characteristics for each subpicture may be signaled in the SPPS instead of the SPS. Tile partitioning within individual subpictures may be signaled in a PPS, but tile groups within the same picture are allowed to reference different PPSs. Once activated, an SPPS may continue throughout a sequence of consecutive AUs in decoding order, but may be deactivated / activated in an AU that is not the start of a CVS. Multiple SPPSs may be active at any point during the decoding process of a single layer bitstream with multiple subpictures, and an SPPS may be shared by different subpictures of an AU. Alternatively, the SPPS and PPS can be merged into one parameter set. For this to happen, all tile groups included in the same subpicture may be constrained to reference the same parameter set resulting from the merge between the SPPS and the PPS.

[0132] The number of bits used for signaling the sub-picture ID may be signaled in the NAL unit header. Such information, when present, aids the sub-picture extraction process in parsing the sub-picture ID value for the start of the NAL unit payload (e.g., the first few bits immediately after the NAL unit header). For such signaling, some of the reserved bits in the NAL unit header may be used to avoid increasing the length of the NAL unit header. The number of bits for such signaling should cover the value of sub-picture-ID-bit-len. For example, four bits out of the seven reserved bits in the NAL unit header of VVC may be used for this purpose.

[0133] When decoding a subpicture, the position of each coding tree block, denoted as vertical CTB position (xCtb) and horizontal CTB position (yCtb), is adjusted to the actual luma sample position in the picture, not the luma sample position in the subpicture. In this way, extraction of the subpictures at the same position from each reference picture can be avoided, since everything is decoded as if it were located in the picture, not in the subpicture. To adjust the position of the coding tree blocks, variables SubpictureXOffset and SubpictureYOffset are derived based on the subpicture positions (subpic_x_offset and subpic_y_offset). The values ​​of the variables are added to the values ​​of the luma sample position x and y coordinates of each coding tree block in the subpicture, respectively. The subpicture extraction process can be defined as follows: The input to the process includes the target subpicture to be extracted. This can be input in the form of a subpicture ID or a subpicture position. When the input is the position of the subpicture, the associated subpicture ID can be resolved by parsing the subpicture information in the SPS. For non-VCL NAL units, the following applies: Syntax elements in the SPS related to picture size and level are updated with the sub-picture size and level information. The following non-VCL NAL units are left unchanged by the extraction: PPS, access unit delimiter (AUD), end of sequence (EOS), end of bitstream (EOB), and any other non-VCL NAL units applicable at or above the picture level. Remaining non-VCL NAL units whose sub-picture ID is not equal to the target sub-picture ID are removed. VCL NAL units whose sub-picture ID is not equal to the target sub-picture ID are also removed.

[0134] The sub-picture nesting SEI message may be used for nesting of AU-level or sub-picture-level SEI messages for a set of sub-pictures. Data carried in the sub-picture nesting SEI message may include buffering period, picture timing, and non-HRD SEI messages. The syntax and semantics of this SEI message may be as follows: For system operation such as an omnidirectional media format (OMAF) environment, a set of sub-picture sequences covering a viewport may be requested and decoded by an OMAF player. Thus, a sequence-level SEI message may carry information of a set of sub-picture sequences that together contain a rectangular or square picture area. The information may be used by the system, and indicates the minimum decoding capability and the bitrate of the set of sub-picture sequences. The information includes the level of the bitstream that contains only the set of sub-picture sequences, the bitrate of the bitstream, and optionally the sub-bitstream extraction process specified for the set of sub-picture sequences.

[0135] The above implementation scheme includes some problems. Signaling picture width and height and / or sub-picture width / height / offset is not efficient. More bits can be saved to signal such information. When sub-picture size and position information is signaled in SPS, PPS includes tile configuration. Furthermore, PPS is allowed to be shared by multiple sub-pictures of the same picture. Therefore, the value range for num_tile_columns_minus1 and num_tile_rows_minus1 should be specified more clearly. Furthermore, the semantics of the flag indicating whether a sub-picture is motion constrained or not is not clearly specified. The level is mandatory signaled for each sub-picture sequence. However, it is not useful to signal the level of a sub-picture when the sub-picture sequence cannot be decoded independently. Furthermore, in some applications, some sub-picture sequences should be decoded and rendered together with at least one other sub-picture sequence. Therefore, it may not be useful to signal the level for a single one of such sub-picture sequences. Furthermore, determining the level value for each sub-picture can be burdensome for the encoder.

[0136] With the introduction of independently decodable sub-picture sequences, scenarios that require independent extraction and decoding of certain regions of a picture may not work based on tile groups. Therefore, explicit signaling of tile group IDs may not be useful. Furthermore, the respective values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag should be the same for all PPSs referenced by the tile group header of the coded picture. This is because the active SPS should not change in the CVS and the value of loop_filter_across_tiles_enabled_flag should be the same for all tiles in a picture for tile-based parallel processing. It should be clearly specified whether to allow a mix of rectangular and raster scan tile groups in a picture. It should also be specified whether to allow sub-pictures that are part of different pictures and use the same sub-picture ID in the CVS to use different tile group modes. The derivation process for temporal luma motion vector prediction may not allow treating sub-picture boundaries as picture boundaries in temporal motion vector prediction (TMVP). Furthermore, the luma sample bilinear interpolation process, the luma sample 8-tap interpolation filtering process, and the chroma sample interpolation process may not be configured to treat sub-picture boundaries as picture boundaries in motion compensation. Also, mechanisms for control of deblocking, SAO, and ALF filtering operations at sub-picture boundaries should be specified.

[0137] With the introduction of independently decodable sub-picture sequences, loop_filter_across_tile_groups_enabled_flag may be less useful, since turning off in-loop filtering operations for parallel processing purposes may also be satisfied by setting loop_filter_across_tile_groups_enabled_flag equal to 0. Furthermore, turning off in-loop filtering operations to enable independent extraction and decoding of specific regions of a picture may also be satisfied by setting loop_filter_across_sub_pic_enabled_flag equal to 0. Thus, further specifying a process for turning off in-loop filtering operations across tile group boundaries based on loop_filter_across_tile_groups_enabled_flag unnecessarily burdens the decoder and wastes bits. Furthermore, the above decoding process may not allow turning off ALF filtering operations across tile boundaries.

[0138] Therefore, this disclosure includes designs for supporting sub-picture based video coding. A sub-picture is a rectangular or square region within a picture that may or may not be independently decoded using the same decoding process as the picture. The description of the technique is based on the Versatile Video Coding (VVC) standard. However, the technique may also be applied to other video codec specifications.

[0139] In some examples, a size unit is signaled for the picture width and height syntax elements and the list of subpicture width / height / offset_x / offset_y syntax elements. All syntax elements are signaled in the format of xxx_minus1. For example, when the size unit is 64 luma samples, a width value of 99 specifies a picture width of 6400 luma samples. The same example applies to others of these syntax elements. In other examples, one or more of the following apply: A size unit may be signaled for the picture width and height syntax elements in the format of xxx_minus1. Such size units signaled for the list of subpicture width / height / offset_x / offset_y syntax elements may also be in the format of xxx_minus1. In other examples, one or more of the following apply: The size units for the lists of picture width and sub-picture width / offset_x syntax elements may be signaled in the form of xxx_minus1. The size units for the lists of picture height and sub-picture height / offset_y syntax elements may be signaled in the form of xxx_minus1. In other examples, one or more of the following apply: The picture width and height syntax elements in the form of xxx_minus1 may be signaled in units of minimum coding units. The sub-picture width / height / offset_x / offset_y syntax elements in the form of xxx_minus1 may be signaled in units of CTUs or CTBs. The sub-picture width for each sub-picture at the right picture border may be derived. The sub-picture height for each sub-picture at the bottom picture border may be derived. All other values ​​of sub-picture width / height / offset_x / offset_y may be signaled in the bitstream. In another example, a mode for signaling the width and height of sub-pictures and their position within the picture may be added for the case where the sub-pictures have a uniform size.Subpictures have a uniform size when they contain the same subpicture rows and subpicture columns. In this mode, the number of subpicture rows, the number of subpicture columns, the width of each subpicture column, and the height of each subpicture row may all be signaled.

[0140] In another example, the signaling of the subpicture width and height may not be included in the PPS. num_tile_columns_minus1 and num_tile_rows_minus1 should be in the range from 0 to an integer value, such as 1024. In another example, when a subpicture that references a PPS has more than one tile, two syntax elements conditioned by a presence flag may be signaled in the PPS. These syntax elements are used to signal the subpicture width and height in units of CTBs and specify the size of all subpictures that reference the PPS.

[0141] In other examples, further information describing individual sub-pictures may also be signaled. A flag such as sub_pic_treated_as_pic_flag[i] may be signaled for each sub-picture sequence to indicate whether the sub-pictures of the sub-picture sequence are treated as pictures in the decoding process for purposes other than in-loop filtering operations. The level to which the sub-picture sequence conforms may only be signaled when sub_pic_treated_as_pic_flag[i] is equal to 1. A sub-picture sequence is the CVS of sub-pictures with the same sub-picture ID. When sub_pic_treated_as_pic_flag[i] is equal to 1, the level of the sub-picture sequence may also be signaled. This can be controlled by a flag for all sub-picture sequences or by one flag for each sub-picture sequence. In other examples, sub-bitstream extraction may be enabled without modifying the VCL NAL units. This can be achieved by removing the explicit tile group ID signaling from the PPS. The semantics of tile_group_address are specified when rect_tile_group_flag is equal to indicate a rectangular tile group. The tile_group_address may contain a tile group index of a tile group within a tile group in a subpicture.

[0142] In another example, the respective values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag shall be the same in all PPSs referenced by the tile group header of the coded picture. Other PPS syntax elements may be different for different PPSs referenced by the tile group header of the coded picture. The value of single_tile_in_pic_flag may be different for different PPSs referenced by the tile group header of the coded picture. In this way, some pictures in the CVS may have only one tile, while some other pictures in the CVS may have multiple tiles. This also allows some sub-pictures of a picture (e.g., very large ones) to have multiple tiles, while other sub-pictures of the same picture (e.g., very small ones) have only one tile.

[0143] In another example, a picture may contain a mix of rectangular and raster scan tile groups. Thus, some subpictures of a picture use rectangular tile group mode while other subpictures use raster scan tile group mode. This flexibility is beneficial for bitstream merge scenarios. Alternatively, the constraint may require that all subpictures of a picture shall use the same tile group mode. Subpictures from different pictures with the same subpicture ID in the CVS may not use different tile group modes. Subpictures from different pictures with the same subpicture ID in the CVS may use different tile group modes.

[0144] In another example, when sub_pic_treated_as_pic_flag[i] for a subpicture is equal to 1, co-located motion vectors for temporal motion vector prediction for the subpicture are constrained to originate from within the boundary of the subpicture. Thus, temporal motion vector prediction for the subpicture is treated as if the subpicture boundary were a picture boundary. Furthermore, to enable treating the subpicture boundary as a picture boundary in motion compensation for subpictures with sub_pic_treated_as_pic_flag[i] equal to 1, clipping operations are specified as part of the luma sample bilinear interpolation process, the luma sample 8-tap interpolation filtering process, and the chroma sample interpolation process.

[0145] In another example, each sub-picture is associated with a signaled flag such as loop_filter_across_sub_pic_enabled_flag. The flag is used to control the in-loop filtering operation at the boundaries of the sub-picture and to control the filtering operation in the corresponding decoding process. The deblocking filter process may not be applied to coding sub-block and transform block edges that coincide with the boundaries of sub-pictures with loop_filter_across_sub_pic_enabled_flag equal to 0. Alternatively, the deblocking filter process is not applied to coding sub-block and transform block edges that coincide with the top or left boundaries of sub-pictures with loop_filter_across_sub_pic_enabled_flag equal to 0. Alternatively, the deblocking filter process is not applied to coding sub-block and transform block edges that coincide with the boundaries of sub-pictures with sub_pic_treated_as_pic_flag[i] equal to 1 or 0. Alternatively, the deblocking filter process is not applied to coding sub-block and transform block edges that coincide with the top or left boundaries of sub-pictures. A clipping operation may be specified to turn off SAO filtering operations across subpicture boundaries when loop_filter_across_sub_pic_enabled_flag for a subpicture is equal to 0. A clipping operation may be specified to turn off ALF filtering operations across subpicture boundaries when loop_filter_across_sub_pic_enabled_flag is equal to 0 for a subpicture. loop_filter_across_tile_groups_enabled_flag may also be removed from the PPS. Thus, when loop_filter_across_tiles_enabled_flag is equal to 0, in-loop filtering operations across tile group boundaries that are not subpicture boundaries are not turned off. Loop filter operations may include deblocking, SAO, and ALF.In another example, the clipping operation is specified to turn off ALF filtering operations across tile boundaries when loop_filter_across_tiles_enabled_flag for the tile is equal to 0.

[0146] One or more of the above examples may be implemented as follows: A subpicture may be defined as a rectangular or square region of one or more tile groups or slices within a picture. The following divisions of processing elements may form a spatial or component-wise partitioning: division of each picture into components, division of each component into CTBs, division of each picture into subpictures, division of each subpicture into tile columns within the subpicture, division of each subpicture into tile rows within the subpicture, division of each tile column within a subpicture into tiles, division of each tile row within a subpicture into tiles, and division of each subpicture into tile groups.

[0147] The process for the CTB raster and tile scan process within a subpicture may be as follows: A list ColWidth[i], for i in the range 0 to num_tile_columns_minus1, specifying the width of the i-th tile column in units of CTB, may be derived as follows:

number

[0148] The list RowHeight[j], for j in the range 0 to num_tile_rows_minus1, specifying the height of the jth tile row in CTB units, is derived as follows:

number

[0149] The list ColBd[i], for i in the range 0 to num_tile_columns_minus1+1 inclusive, specifying the position of the i-th tile column boundary in CTB units, is derived as follows:

number

[0150] The list RowBd[j], for j in the range 0 to num_tile_rows_minus1+1, specifying the position of the jth tile row boundary in CTB units, is derived as follows:

number

[0151] The list CtbAddrRsToTs[ctbAddrRs] for ctbAddrRs in the range 0 to SubPicSizeInCtbsY-1, inclusive, which specifies the conversion from CTB addresses in the CTB raster scan of the subpicture to CTB addresses in the subpicture tile scan, is derived as follows:

number

[0152] The list CtbAddrTsToRs[ctbAddrTs] for ctbAddrTs in the range 0 to SubPicSizeInCtbsY-1, which specifies the conversion from CTB addresses in a tile scan to CTB addresses in a subpicture CTB raster scan, is derived as follows:

number

[0153] The list TileId[ctbAddrTs] for ctbAddrTs in the range 0 to SubPicSizeInCtbsY-1, which specifies the conversion from CTB addresses to tile IDs in a subpicture tile scan, is derived as follows:

number

[0154] The list NumCtusInTile[tileIdx] for tileIdx in the range from 0 to NumTilesInSubPic - 1, which specifies the conversion from the tile index to the number of CTUs in the tile, is derived as follows.

Number

[0155] The list FirstCtbAddrTs[tileIdx] for tileIdx in the range from 0 to NumTilesInSubPic - 1, which specifies the conversion from the tile ID to the CTB address in the tile scan of the first CTB in the tile, is derived as follows.

Number

[0156] The value of ColumnWidthInLumaSamples[i] that specifies the width of the i-th tile column in units of luma samples is set equal to ColWidth[i] << CtbLog2SizeY for i in the range from 0 to num_tile_columns_minus1. The value of RowHeightInLumaSamples[j] that specifies the height of the j-th tile row in units of luma samples is set equal to RowHeight[j] << CtbLog2SizeY for j in the range from 0 to num_tile_rows_minus1.

[0157] The syntax of an exemplary sequence parameter set RBSP is as follows.

Table 1

[0158] The syntax of an exemplary picture parameter set RBSP is as follows. [Table 2] TIFF2024019407000015.tif34169

[0159] The syntax of an exemplary general tile group header is as follows: [Table 3]

[0160] The syntax of an exemplary coding tree unit is as follows: [Table 4]

[0161] The semantics of an exemplary sequence parameter set RBSP are as follows:

[0162] bit_depth_chroma_minus8 specifies the bit depth of the samples in the chroma array BitDepthC, and the value of the chroma quantization parameter range offset QpBdOffsetC is as follows:

number

[0163] num_sub_pics_minus1 plus 1 specifies the number of subpictures in each coded picture in the CVS. The value of num_sub_pics_minus1 shall be in the range 0 to 1024, inclusive. sub_pic_id_len_minus1 plus 1 specifies the number of bits used to represent the syntax element sub_pic_id[i] in the SPS and the syntax element tile_group_sub_pic_id in the tile group header. The value of sub_pic_id_len_minus1 shall be in the range of Ceil(Log2(num_sub_pic_minus1+1)-1 to 9, inclusive. sub_pic_level_present_flag is set to 1 to specify that the syntax element sub_pic_level_idc[i] may be present. sub_pic_level_present_flag is set to 0 to specify that the syntax element sub_pic_level_idc[i] is not present. sub_pic_id[i] specifies the sub-picture ID of the i-th sub-picture of the respective coded picture in the CVS. The length of sub_pic_id[i] is sub_pic_id_len_minus1+1 bits.

[0164] sub_pic_treated_as_pic_flag[i] is set equal to 1 to specify that the i-th subpicture of each coded picture in the CVS is treated as a picture in the decoding process, except for in-loop filtering operations. sub_pic_treated_as_pic_flag[i] is set equal to 0 to specify that the i-th subpicture of each coded picture in the CVS is not treated as a picture in the decoding process, except for in-loop filtering operations. sub_pic_level_idc[i] indicates the level to which the i-th subpicture sequence conforms, where the i-th subpicture sequence consists only of VCL NAL units of subpictures with subpicture IDs equal to sub_pic_id[i] in the CVS and their associated non-VCL NAL units. sub_pic_x_offset[i] specifies the horizontal offset, in units of luma samples, of the top-left corner luma sample of the i-th subpicture relative to the top-left corner luma sample of each picture in the CVS. When not present, the value of sub_pic_x_offset[i] is inferred to be equal to 0. sub_pic_y_offset[i] specifies the vertical offset, in units of luma samples, of the top-left luma sample of the i-th subpicture relative to the top-left luma sample of each picture in the CVS. When not present, the value of sub_pic_y_offset[i] is inferred to be equal to 0. sub_pic_width_in_luma_samples[i] specifies the width, in units of luma samples, of the i-th subpicture of each picture in the CVS. When the sum of sub_pic_x_offset[i] and sub_pic_width_in_luma_samples[i] is less than pic_width_in_luma_samples, the value of sub_pic_width_in_luma_samples[i] shall be an integer multiple of CtbSizeY. When not present, the value of sub_pic_width_in_luma_samples[i] is inferred to be equal to pic_width_in_luma_samples.sub_pic_height_in_luma_samples[i] specifies the height, in units of luma samples, of the i-th subpicture of each picture in the CVS. When the sum of sub_pic_y_offset[i] and sub_pic_height_in_luma_samples[i] is less than pic_height_in_luma_samples, the value of sub_pic_height_in_luma_samples[i] shall be an integer multiple of CtbSizeY. When not present, the value of sub_pic_height_in_luma_samples[i] is inferred to be equal to pic_height_in_luma_samples.

[0165] For bitstream conformance, the following constraints apply: For any integer values ​​of i and j, when i is equal to j, the values ​​of sub_pic_id[i] and sub_pic_id[j] shall not be the same. For any two subpictures subpicA and subpicB, when the subpicture ID of subpicA is smaller than the subpicture ID of subpicB, any coded tile group NAL unit of subPicA shall follow any coded tile group NAL unit of subPicB in decoding order. The shape of the subpictures shall be such that each subpicture, when decoded, has its entire left and entire top boundary consisting of either the picture boundary or the boundary of a previously decoded subpicture.

[0166] The list SubPicIdx[spId], for spId values ​​equal to sub_pic_id[i] for i in the range 0 to num_sub_pics_minus1, which specifies the conversion from subpicture IDs to subpicture indices, is derived as follows:

number

[0167] log2_max_pic_order_cnt_lsb_minus4 specifies the value of the variable MaxPicOrderCntLsb used in the decoding process for picture order count as follows:

number

[0168] The semantics of an exemplary picture parameter set RBSP are as follows:

[0169] When present, the respective values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag shall be the same for all PPSs referenced by the tile group header of the coded picture. pps_pic_parameter_set_id identifies the PPS referenced by other syntax elements. The value of pps_pic_parameter_set_id shall be in the range from 0 to 63, inclusive. pps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for the active SPS. The value of pps_seq_parameter_set_id shall be in the range from 0 to 15, inclusive. loop_filter_across_sub_pic_enabled_flag shall be set equal to 1 to specify that in-loop filtering operations may be performed across sub-picture boundaries that reference the PPS. loop_filter_across_sub_pic_enabled_flag is set equal to 0 to specify that in-loop filtering operations are not performed across sub-picture boundaries that reference a PPS.

[0170] single_tile_in_sub_pic_flag is set equal to 1 to specify that there is only one tile in each subpicture that references the PPS. single_tile_in_sub_pic_flag is set equal to 0 to specify that there is more than one tile in each subpicture that references the PPS. num_tile_columns_minus1 plus 1 specifies the number of tile columns that partition the subpicture. num_tile_columns_minus1 shall be in the range from 0 to 1024, inclusive. When not present, the value of num_tile_columns_minus1 is inferred to be equal to 0. num_tile_rows_minus1 plus 1 specifies the number of tile rows that partition the subpicture. num_tile_rows_minus1 shall be in the range from 0 to 1024, inclusive. When not present, the value of num_tile_rows_minus1 is inferred to be equal to 0. The variable NumTilesInSubPic is set equal to (num_tile_columns_minus1+1)*(num_tile_rows_minus1+1). When single_tile_in_sub_pic_flag is equal to 0, NumTilesInSubPic shall be greater than 1.

[0171] uniform_tile_spacing_flag is set equal to 1 to specify that the tile column borders, and similarly the tile row borders, are uniformly distributed among the subpictures. uniform_tile_spacing_flag is set equal to 0 to specify that the tile column borders, and similarly the tile row borders, are not uniformly distributed among the subpictures, but are explicitly signaled using the syntax elements tile_column_width_minus1[i] and tile_row_height_minus1[i]. When not present, the value of uniform_tile_spacing_flag is inferred to be equal to 1. tile_column_width_minus1[i] plus 1 specifies the width of the i-th tile column in units of CTBs. tile_row_height_minus1[i] plus 1 specifies the height of the i-th tile row in units of CTBs. single_tile_per_tile_group is set equal to 1 to specify that each tile group that references this PPS contains one tile. single_tile_per_tile_group is set equal to 0 to specify that the tile group that references this PPS may contain more than one tile.

[0172] rect_tile_group_flag is set equal to 0 to specify that the tiles in each tile group of the subpicture are in raster scan order and tile group information is not signaled in the PPS. rect_tile_group_flag is set equal to 1 to specify that the tiles in each tile group cover a rectangular or square area of ​​the subpicture and tile group information is signaled in the PPS. When single_tile_per_tile_group_flag is set to 1, rect_tile_group_flag is inferred to be equal to 1. num_tile_groups_in_sub_pic_minus1 plus 1 specifies the number of tile groups in each subpicture that references the PPS. The value of num_tile_groups_in_sub_pic_minus1 shall be in the range from 0 to NumTilesInSubPic-1, inclusive. When not present and single_tile_per_tile_group_flag is equal to 1, the value of num_tile_groups_in_sub_pic_minus1 is inferred to be equal to NumTilesInSubPic-1.

[0173] top_left_tile_idx[i] specifies the tile index of the tile located in the top-left corner of the i-th tile group of the subpicture. The value of top_left_tile_idx[i] shall not be equal to the value of top_left_tile_idx[j] for any i not equal to j. When not present, the value of top_left_tile_idx[i] is inferred to be equal to i. The length of the top_left_tile_idx[i] syntax element is Ceil(Log2(NumTilesInSubPic) bits. bottom_right_tile_idx[i] specifies the tile index of the tile located in the bottom right corner of the i-th tile group of the subpicture. When single_tile_per_tile_group_flag is set to 1, bottom_right_tile_idx[i] is inferred to be equal to top_left_tile_idx[i]. The length of the bottom_right_tile_idx[i] syntax element is Ceil(Log2(NumTilesInSubPic)) bits.

[0174] It is a bitstream conformance requirement that any particular tile is contained in only one tile group. The variable NumTilesInTileGroup[i], which specifies the number of tiles in the i-th tile group of a subpicture, and related variables are derived as follows:

number

[0175] loop_filter_across_tiles_enabled_flag is set equal to 1 to specify that in-loop filtering operations may be performed across tile boundaries in subpictures that reference a PPS. loop_filter_across_tiles_enabled_flag is set equal to 0 to specify that in-loop filtering operations are not performed across tile boundaries in subpictures that reference a PPS. In-loop filtering operations include deblocking filter, sample adaptive offset filter, and adaptive loop filter operations. When not present, the value of loop_filter_across_tiles_enabled_flag is inferred to be equal to 1. num_ref_idx_default_active_minus1[i] plus 1 specifies an estimate of the variable NumRefIdxActive[0] for a P or B tile group that has num_ref_idx_active_override_flag equal to 0 when i is equal to 0, and specifies an estimate of NumRefIdxActive[1] for a B tile group that has num_ref_idx_active_override_flag equal to 0 when i is equal to 1. The value of num_ref_idx_default_active_minus1[i] shall be in the range 0 to 14, inclusive.

[0176] The semantics of an exemplary general tile group header are as follows: When present, the respective values ​​of the tile group header syntax elements tile_group_pic_order_cnt_lsb and tile_group_temporal_mvp_enabled_flag shall be the same in all tile group headers of a coded picture. When present, the value of tile_group_pic_parameter_set_id shall be the same in all tile group headers of a coded subpicture. tile_group_pic_parameter_set_id specifies the value of pps_pic_parameter_set_id for the PPS in use. The value of tile_group_pic_parameter_set_id shall be in the range 0 to 63, inclusive. It is a bitstream conformance requirement that the value of TemporalId of the current picture shall be greater than or equal to the value of TemporalId of each PPS referenced by the tile group of the current picture. tile_group_sub_pic_id identifies the subpicture to which the tile group belongs. The length of tile_group_sub_pic_id is sub_pic_id_len_minus1 + 1 bits. The value of tile_group_sub_pic_id shall be the same for all tile group headers of a coded sub-picture.

[0177] The variables SubPicWidthInCtbsY, SubPicHeightInCtbsY and SubPicSizeInCtbsY are derived as follows:

number

[0178] The following variables: a list ColWidth[i] for i in the range 0 to num_tile_columns_minus1 inclusive that specifies the width of the ith tile column in CTBs; a list RowHeight[j] for j in the range 0 to num_tile_rows_minus1 inclusive that specifies the height of the jth tile row in CTBs; a list ColBd[i] for i in the range 0 to num_tile_columns_minus1+1 inclusive that specifies the position of the ith tile column boundary in CTBs; A list RowBd[j] for j in the range 0 to num_tile_rows_minus1+1, inclusive, specifying the location of the tile row boundary. A list CtbAddrRsToTs[ctbAddrRs] for ctbAddrRs in the range 0 to SubPicSizeInCtbsY-1, inclusive, specifying the conversion from the CTB address in the subpicture CTB raster scan to the CTB address in the subpicture tile scan. A list of ctbAddrTsToRs[ctbAddrTs] for ctbAddrTs in the range 0 to SubPicSizeInCtbsY-1 inclusive, specifying the conversion to CTB addresses; a list of ctbAddrTsTileId[ctbAddrTs] for ctbAddrTs in the range 0 to SubPicSizeInCtbsY-1 inclusive, specifying the conversion from CTB addresses to tile IDs in subpicture tile scans; a list of ctbAddrTsTileId[ctbAddrTs] for ctbAddrTs in the range 0 to SubPicSizeInCtbsY-1 inclusive, specifying the conversion from tile indexes to the number of CTUs in a tile; A list NumCtusInTile[tileIdx], for the range tileIdx; a list FirstCtbAddrTs[tileIdx], for tileIdx, in the range 0 to NumTilesInSubPic-1, inclusive, that specifies the conversion from a tile ID to the CTB address in the tile scan of the first CTB in the tile; a list ColumnWidthInLumaSamples[i], for i, in the range 0 to num_tile_columns_minus1, inclusive, that specifies the width of the i-th tile column in units of luma samples;and a list RowHeightInLumaSamples[j], for j in the range 0 to num_tile_rows_minus1, inclusive, specifying the height of the jth tile row in units of luma samples, derived by invoking the CTB raster and tile scan conversion process.

[0179] The values ​​of ColumnWidthInLumaSamples[i] for i in the range of 0 to num_tile_columns_minus1, inclusive, and RowHeightInLumaSamples[j] for j in the range of 0 to num_tile_rows_minus1, inclusive, shall all be greater than 0. The variables SubPicLeftBoundaryPos, SubPicTopBoundaryPos, SubPicRightBoundaryPos, and SubPicBotBoundaryPos are derived as follows.

number

[0180] For each tile with index i=0.NumTilesInSubPic-1 in the current subpicture, the variables TileLeftBoundaryPos[i], TileTopBoundaryPos[i], TileRightBoundaryPos[i] and TileBotBoundaryPos[i] are derived as follows:

number

[0181] tile_group_address specifies the tile address of the first tile in the tile group. When not present, the value of tile_group_address is inferred to be equal to 0. If rect_tile_group_flag is equal to 0, the following applies: the tile address is a tile ID, the length of tile_group_address is Ceil(Log2(NumTilesInSubPic)) bits, and the value of tile_group_address shall be in the range from 0 to NumTilesInSubPic-1 inclusive. Otherwise (rect_tile_group_flag is equal to 1), the following applies: the tile address is a tile group index of a tile group within a tile group in a subpicture, the length of tile_group_address is Ceil(Log2(num_tile_groups_in_sub_pic_minus1+1)) bits, and the value of tile_group_address shall be in the range from 0 to num_tile_groups_in_sub_pic_minus1 inclusive.

[0182] It is a requirement for bitstream conformance that the following constraints apply: The value of tile_group_address shall not be equal to the value of tile_group_address of any other coded tile group NAL unit of the same coded subpicture. The tile groups of a subpicture shall be in increasing order of their tile_group_address values. The shape of the tile groups of a subpicture shall be such that each tile, when decoded, shall have its entire left and entire top boundary consisting of either the subpicture boundary or the boundary of a previously decoded tile.

[0183] num_tiles_in_tile_group_minus1, when present, specifies the number of tiles in the tile group minus 1. The value of num_tiles_in_tile_group_minus1 shall be in the range 0 to NumTilesInSubPic-1 inclusive. When not present, the value of num_tiles_in_tile_group_minus1 is inferred to be equal to 0. The variables NumTilesInCurrTileGroup, which specifies the number of tiles in the current tile group, and TgTileIdx[i], which specifies the tile index of the i-th tile in the current tile group, are derived as follows:

number

[0184] An exemplary derivation process for temporal luma motion vector prediction is as follows: The variables mvLXCol and availableFlagLXCol are derived as follows: If tile_group_temporal_mvp_enabled_flag is equal to 0, then both components of mvLXCol are set equal to 0, and availableFlagLXCol is set equal to 0. Otherwise (tile_group_temporal_mvp_enabled_flag is equal to 1), the following ordered steps apply: The bottom right co-located motion vector, as well as the bottom and right boundary sample positions are derived as follows:

number

[0185] An exemplary luma sample bilinear interpolation process is as follows: The luma position in whole sample units (xInti, yInti) is derived as follows, for i=0..1: If sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following holds:

number

number

[0186] An exemplary luma sample 8-tap interpolation filtering process is as follows: The luma position in whole sample units (xInti, yInti) is derived as follows, for i=0..7: If sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following holds:

number

number

[0187] An exemplary chroma sample interpolation process is as follows: The variable xOffset is set equal to (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC. The chroma position in whole sample units (xInti, yInti) is derived for i=0..3 as follows:

[0188] If sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following is true:

number

number

[0189] An exemplary deblocking filter process is as follows: The deblocking filter process is applied to all coding sub-block and transform block edges of a picture except for the following types of edges: edges at picture boundaries, edges that coincide with sub-picture boundaries with loop_filter_across_sub_pic_enabled_flag equal to 0, edges that coincide with tile boundaries with loop_filter_across_tiles_enabled_flag equal to 0, edges that coincide with top or left boundaries of or within a tile group with tile_group_deblocking_filter_disabled_flag equal to 1, edges that do not correspond to an 8x8 sample grid boundary of the component considered, edges in chroma components where both sides of the edge use inter prediction, edges of chroma transform blocks that are not edges of the associated transform unit, and edges that cross a luma transform block of a coding unit with an IntraSubPartitionsSplit value not equal to ISP_NO_SPLIT.

[0190] An exemplary deblocking filter process in one direction is as follows. For each coding unit having a coding block width log2CbW, a coding block height log2CbH, and the position (xCb, yCb) of the top-left sample of the coding block, when edgeType is equal to EDGE_VER and xCb % 8 is equal to 0, or when edgeType is equal to EDGE_HOR and yCb % 8 is equal to 0, the edge is filtered by the steps in the following order. The coding block width nCbW is set equal to 1 << log2CbW, and the coding block height nCbH is set equal to 1 << log2CbH. The variable filterEdgeFlag is derived as follows. When edgeType is equal to EDGE_VER and one or more of the following conditions are true, filterEdgeFlag is set equal to 0. The left boundary of the current coding block is the left boundary of the picture. The left boundary of the current coding block is the left or right boundary of the sub-picture and loop_filter_across_sub_pic_enabled_flag is equal to 0. The left boundary of the current coding block is the left boundary of the tile and loop_filter_across_tiles_enabled_flag is equal to 0. Otherwise, when edgeType is equal to EDGE_HOR and one or more of the following conditions are true, the variable filterEdgeFlag is set equal to 0. The top boundary of the current luma coding block is the top boundary of the picture. The top boundary of the current coding block is the top or bottom boundary of the sub-picture and loop_filter_across_sub_pic_enabled_flag is equal to 0. The top boundary of the current coding block is the top boundary of the tile and loop_filter_across_tiles_enabled_flag is equal to 0. Otherwise, filterEdgeFlag is set equal to 1.

[0191] An exemplary CTB modification process is as follows: For all sample positions (xSi, ySj) and (xYi, yYj), for i=0..nCtbSw-1 and j=0..nCtbSh-1, depending on the values ​​of pcm_loop_filter_disabled_flag, pcm_flag[xYi][yYj], and cu_transquant_bypass_flag of the coding unit that contains the coding block covering recPicture[xSi][ySj], the following is true: If one or more of the following conditions are true for all sample positions (xSik', ySjk') and (xYik', yYjk'), for k=0..1, edgeIdx is set equal to 0: The sample at position (xSik', ySjk') is outside the picture boundary. The sample at position (xSik', ySjk') belongs to a different subpicture, and loop_filter_across_sub_pic_enabled_flag in the tile group to which sample recPicture[xSi][ySj] belongs is equal to 0. loop_filter_across_tiles_enabled_flag is equal to 0, and the sample at position (xSik', ySjk') belongs to a different tile.

[0192] An example coding tree block filtering process for luma samples is as follows: For derivation of a filtered reconstructed luma sample alfPictureL[x][y], each reconstructed luma sample in the current luma coding tree block recPictureL[x][y] is filtered by x,y=0..CtbSizeY-1 as follows: Each position (hx,vy) of a corresponding luma sample (x,y) in a given array of luma samples recPicture is derived as follows: If loop_filter_across_tiles_enabled_flag for a tile tileA that contains a luma sample at position (hx,vy) is equal to 0, then the following applies, where variable tileIdx is the tile index of tileA:

number

number

number

[0193] An exemplary derivation process for the ALF transpose and filter index for a luma sample is as follows: For each position (hx,vy) of a corresponding luma sample (x,y) in a given array of luma samples, recPicture, the following is derived: If loop_filter_across_tiles_enabled_flag for the tile tileA that contains the luma sample at position (hx,vy) is equal to 0, then let tileIdx be the tile index of tileA, then the following holds:

number

number

number

[0194] An example coding tree block filtering process for chroma samples is as follows: For derivation of a filtered reconstructed chroma sample alfPicture[x][y], each reconstructed chroma sample in the current chroma coding tree block recPicture[x][y] is filtered by x,y=0..ctbSizeC-1 as follows: Each position (hx,vy) of a corresponding chroma sample (x,y) in a given array of chroma samples recPicture is derived as follows: If loop_filter_across_tiles_enabled_flag for a tile tileA containing a chroma sample at position (hx,vy) is equal to 0, then the following holds: Let tileIdx be the tile index of tileA.

number

number

number

[0195] The sum of the variables is derived as follows:

number

number

[0196] FIG. 12 is a schematic diagram of an example video coding device 1200. The video coding device 1200 is suitable for implementing the disclosed examples / embodiments as described herein. The video coding device 1200 includes a downstream port 1220, an upstream port 1250, and / or a transceiver unit (Tx / Rx) 1210 including a transmitter and / or a receiver for communicating data upstream and / or downstream over a network. The video coding device 1200 also includes a processor 1230 including a logic unit and / or a central processing unit (CPU) for processing data, and a memory 1232 for storing data. The video coding device 1200 may also include electrical, optical-to-electrical (OE), electrical-to-optical (EO), and / or wireless communication components coupled to the upstream port 1250 and / or the downstream port 1220 for communicating data over an electrical, optical, or wireless communication network. Video coding device 1200 may also include input and / or output (I / O) devices 1260 for communicating data to and from a user. I / O devices 1260 may include output devices such as a display for displaying video data, speakers for outputting audio data, etc. I / O devices 1260 may also include input devices such as a keyboard, mouse, trackball, etc., and / or corresponding interfaces for interacting with such output devices.

[0197] The processor 1230 is implemented by hardware and software. The processor 1230 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 1230 communicates with the downstream port 1220, the Tx / Rx 1210, the upstream port 1250, and the memory 1232. The processor 1230 includes a coding module 1214. The coding module 1214 may implement the disclosed embodiments described herein, such as the methods 100, 1300, and 1400, using the in-loop filter 1000, the bitstream 1100, the picture 500, and / or the current block 801 and / or 901 that may be coded according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 based on the candidate list generated according to the pattern 900. Coding module 1214 may also implement any other method / mechanism described herein. Furthermore, coding module 1214 may implement codec system 200, encoder 300 and / or decoder 400. For example, coding module 1214 may implement the first, second, third, fourth, fifth and / or sixth example implementations as described above. Thus, coding module 1214 allows video coding device 1200 to provide additional functionality and / or coding efficiency when coding video data. Thus, coding module 1214 improves the functionality of video coding device 1200 and addresses problems specific to video coding techniques. Furthermore, coding module 1214 effects the transformation of video coding device 1200 into a different state.Alternatively, the coding module 1214 may be embodied as instructions stored in memory 1232 and executed by the processor 1230 (eg, as a computer program product stored on a non-transitory medium).

[0198] Memory 1232 may include one or more memory types such as a disk, a tape drive, a solid state drive, read only memory (ROM), random access memory (RAM), flash memory, ternary content-addressable memory (TCAM), static random-access memory (SRAM), etc. Memory 1232 may store such programs when such programs are selected for execution and may be used as an overflow data storage device to store instructions and data read during program execution.

[0199] 13 is a flowchart of an example method 1300 for encoding a video sequence of a sub-picture, such as sub-picture 510 of picture 500, into a bitstream, such as bitstream 1100. Method 1300 may be used by an encoder, such as codec system 200, encoder 300 and / or video coding device 1200, when performing method 100 for encoding current block 801 and / or 901 according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 on a candidate list generated by using in-loop filter 1000 and / or according to pattern 900.

[0200] Method 1300 may begin when an encoder receives a video sequence including multiple pictures and decides to encode the video sequence into a bitstream based on, for example, user input. In step 1301, the encoder partitions a picture into sub-pictures. In step 1303, the encoder determines a sub-picture width, a sub-picture height, and a sub-picture offset.

[0201] In step 1305, the encoder codes the subpicture width, subpicture height, and subpicture offset into the bitstream. The subpicture width, height, and offset may be coded into the bitstream in the SPS. The subpicture width and subpicture height are coded in units of CTBs. A CTB includes a predefined number of samples, and thus, a CTB describes a predefined unit size in a picture and / or a subpicture. In some examples, the subpicture width is coded into the bitstream as subpic_width_minus1, where subpic_width_minus1 is one CTB less than the number of CTBs in the subpicture width. Furthermore, the subpicture height may be coded into the bitstream as subpic_height_minus1, where subpic_height_minus1 is one CTB less than the number of CTBs in the subpicture height. In this way, the subpicture width or height can be omitted from the bitstream and estimated if the width or height is one CTB, respectively. This saves bits and therefore increases coding efficiency. Furthermore, signaling the width and height in minus-one format may also reduce the coding size of the width and height in the bitstream by one bit in some cases, thus also increasing the coding efficiency for this reason. The subpicture offset may be specified as the vertical position of the top-left CTU of the subpicture and the horizontal position of the top-left CTU of the subpicture. For example, the subpicture offset may be specified as the vertical and horizontal difference between the top-left CTU of the picture and the top-left CTU of the subpicture. The vertical and horizontal positions of the top CTU may also be signaled in units of CTBs. In some examples, the vertical position of the top-left CTU of the subpicture may be coded into the bitstream as subpic_ctu_top_left_y. Furthermore, the vertical position of the top-left CTU of the subpicture may be coded into the bitstream as subpic_ctu_top_left_x.

[0202] In step 1307, the encoder encodes the coding blocks of the sub-picture into a bitstream, and in step 1309, the encoder stores the bitstream for communication to a decoder.

[0203] 14 is a flowchart of an example method 1400 for decoding a video sequence including a sub-picture, such as sub-picture 510 of picture 500, from a bitstream, such as bitstream 1100. Method 1400 may be used by a decoder, such as codec system 200, decoder 400 and / or video coding device 1200, when performing method 100 for decoding current block 801 and / or 901 according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 on a candidate list generated by using in-loop filter 1000 and / or according to pattern 900.

[0204] Method 1400 may begin when a decoder begins receiving a bitstream of coded data representing a video sequence, for example as a result of method 1300. In step 1401, the decoder receives a bitstream that includes pictures that include / are partitioned into sub-pictures.

[0205] In step 1403, the decoder obtains the subpicture width, subpicture height, and subpicture offset from the bitstream. The subpicture width, height, and offset may be obtained from the SPS in the bitstream. The subpicture width and subpicture height are included in the bitstream in units of CTBs. A CTB includes a predefined number of samples, and thus, a CTB describes a predefined unit size in a picture and / or a subpicture. In some examples, the subpicture width is obtained from the bitstream in a variable subpic_width_minus1, which is one CTB less than the number of CTBs in the subpicture width. Furthermore, the subpicture height may be obtained from the bitstream in a variable subpic_height_minus1, which is one CTB less than the number of CTBs in the subpicture height. In this way, the subpicture width or height can be omitted and estimated from the bitstream if the width or height is one CTB, respectively. This saves bits and therefore increases coding efficiency. Furthermore, signaling the width and height in minus-one format may also reduce the coding size of the width and height in the bitstream by one bit in some cases, thus also increasing the coding efficiency for this reason. The subpicture offset may be specified as the vertical position of the top-left CTU of the subpicture and the horizontal position of the top-left CTU of the subpicture. For example, the subpicture offset may be specified in the bitstream as the vertical and horizontal difference between the top-left CTU of the picture and the top-left CTU of the subpicture. The vertical and horizontal positions of the top CTU may also be signaled in units of CTBs. In some examples, the vertical position of the top-left CTU of the subpicture may be obtained from the bitstream in the variable subpic_ctu_top_left_y. Furthermore, the vertical position of the top-left CTU of the subpicture may be obtained from the bitstream in the variable subpic_ctu_top_left_x.

[0206] In step 1405, the decoder decodes the coding blocks of the subpicture based on the subpicture width, subpicture height, and subpicture offset. For example, the decoder can determine the size and location of the subpicture based on the subpicture width, height, and offset. The decoder can also use this information to position the subpicture at a specified location (e.g., relative to the picture), include appropriate slices within the subpicture, and include the correct coding blocks in the slices before decoding the blocks to reconstruct the picture and / or subpicture. In step 1407, the decoder can forward the decoded coding blocks of the subpicture for display as part of the decoded video sequence.

[0207] 15 is a schematic diagram of an example system 1500 for coding a video sequence of an image, such as picture 500, including a sub-picture, such as sub-picture 510, in a bitstream, such as bitstream 1100. System 1500 may be implemented by an encoder and a decoder, such as codec system 200, encoder 300, decoder 400 and / or video coding device 1200. Furthermore, system 1500 may be used when implementing methods 100, 1300 and / or 1400 for coding a current block 801 and / or 901 according to unidirectional inter prediction 600 and / or bidirectional inter prediction 700 by using an in-loop filter 1000 and / or on a candidate list generated according to pattern 900.

[0208] The system 1500 includes a video encoder 1502. The video encoder 1502 includes a partition module 1505 for partitioning a picture into sub-pictures. The video encoder 1502 further includes a determination module 1506 for determining a sub-picture width and a sub-picture height. The video encoder 1502 further includes an encoding module 1507 for encoding the sub-picture width and the sub-picture height in units of CTBs into a bitstream. Furthermore, the encoding module 1507 is for encoding coding blocks of the sub-picture into the bitstream. The video encoder 1502 further includes a storage module 1508 for storing the bitstream for communication towards a decoder. The video encoder 1502 further includes a transmission module 1509 for transmitting the bitstream towards a video decoder 1510. The video encoder 1502 may be further configured to perform any of the steps of the method 1300.

[0209] The system 1500 also includes a video decoder 1510. The video decoder 1510 includes a receiving module 1511 for receiving a bitstream including a picture including a sub-picture. The video decoder 1510 further includes an obtaining module 1512 for obtaining a sub-picture width in units of CTBs and a sub-picture height in units of CTBs from the bitstream. The video decoder 1510 further includes a decoding module 1513 for decoding a coding block of the sub-picture based on the sub-picture width and the sub-picture height. The video decoder 1510 further includes a transport module 1515 for transporting the coding block of the sub-picture for display as part of the decoded video sequence. The video decoder 1510 may be further configured to perform any of the steps of the method 1400.

[0210] A first component is directly coupled to a second component when there are no intervening components, other than lines, traces, or other media between the first and second components. A first component is indirectly coupled to a second component when there are intervening components, other than lines, traces, or other media between the first and second components. The term "coupled" and variations thereof include both directly coupled and indirectly coupled. Use of the term "about" means a range that includes ±10% of the succeeding number, unless otherwise stated.

[0211] It should also be understood that the steps of the exemplary methods described herein do not necessarily have to be performed in the order described, and the order of steps of such methods should be understood to be merely exemplary. Similarly, additional steps may be included in such methods, and certain steps may be omitted or combined in methods consistent with various embodiments of the present disclosure.

[0212] Although several embodiments are provided in this disclosure, it will be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the disclosure. The examples should be considered illustrative and not limiting, and the intention is not to be limited to the details given herein. For example, various elements or components may be combined or integrated into other systems, or certain features may be omitted or not implemented.

[0213] Additionally, the techniques, systems, subsystems, and methods described and illustrated in various embodiments as separate or separate may be combined or integrated with other systems, components, techniques, or methods without departing from the scope of the present disclosure. Other examples of variations, substitutions, and alterations will be ascertainable by one of ordinary skill in the art and may be made without departing from the spirit and scope of the present disclosure.

Claims

1. 1. A method implemented in a decoder, comprising: receiving a bitstream containing coded data of a picture including a sub-picture; obtaining a width of the sub-picture in units of coding tree blocks (CTBs) and a height of the sub-picture in units of CTBs from the bitstream; obtaining a sub-picture identification (ID) of the sub-picture from the bitstream; decoding coding blocks of the subpicture based on the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture; A method comprising:

2. 2. The method of claim 1, further comprising: obtaining a flag for the subpicture from the bitstream, the flag equal to 1 specifying that the subpicture is to be treated as a picture in the decoding process except for in-loop filtering operations, and the flag equal to 0 specifying that the subpicture is not to be treated as a picture in the decoding process except for in-loop filtering operations.

3. The method of claim 2 , wherein the flag is denoted as sub_pic_treated_as_pic_flag.

4. 4. The method of claim 1, wherein the width of the subpicture is stored in the bitstream as subpic_width_minus1 and the height of the subpicture is stored in the bitstream as subpic_height_minus1.

5. The method of claim 1 , wherein the width, height and sub-picture ID of the sub-picture are obtained from a sequence parameter set (SPS) in the bitstream.

6. 1. A method implemented in an encoder, comprising: partitioning a picture into sub-pictures; determining the width of the sub-picture and the height of the sub-picture; encoding the width of the sub-picture and the height of the sub-picture into a bitstream in units of coding tree blocks (CTBs); encoding a sub-picture identification (ID) of said sub-picture into said bitstream; encoding the coding blocks of the sub-picture into the bitstream; A method comprising:

7. 7. The method of claim 6, further comprising: encoding a flag for the subpicture into the bitstream, the flag being set equal to 1 to specify that the subpicture is treated as a picture in the decoding process except for in-loop filtering operations, and the flag being set equal to 0 to specify that the subpicture is not treated as a picture in the decoding process except for in-loop filtering operations.

8. 8. The method of claim 6 or 7, wherein the width of the subpicture is coded into the bitstream as subpic_width_minus1 and the height of the subpicture is coded into the bitstream as subpic_height_minus1.

9. The method of claim 6 , wherein the width, height and sub-picture ID of the sub-picture are coded in a sequence parameter set (SPS) of the bitstream.

10. 1. A video decoding device, comprising:

6. A video decoding device comprising: a processor; a receiver coupled to the processor; a memory coupled to the processor; and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform the method of any one of claims 1 to 5.

11. A video encoding device, comprising:

10. A video encoding device comprising: a processor; a receiver coupled to the processor; a memory coupled to the processor; and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform the method of any one of claims 6 to 9.

12. A non-transitory computer readable medium comprising computer executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 5.

13. A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by a processor, cause the processor to perform a method according to any one of claims 6 to 9.

14. A decoder comprising: a receiving unit configured to receive a bitstream including coded data of a picture including a sub-picture; an acquisition unit configured to acquire a width of the sub-picture in units of coding tree blocks (CTBs) and a height of the sub-picture in units of CTBs from the bitstream, the acquisition unit being further configured to acquire a sub-picture identification (ID) of the sub-picture from the bitstream; a decoding unit configured to decode a coding block of the subpicture based on the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture; and A decoder containing

15. Decoder according to claim 14, wherein the decoder is further configured to perform a method according to any one of claims 2 to 5.

16. 1. An encoder comprising: a partitioning unit configured to partition a picture into sub-pictures; a determining unit configured to determine a width of the sub-picture and a height of the sub-picture; a coding unit configured to encode the width of the sub-picture and the height of the sub-picture into a bitstream in units of coding tree blocks (CTBs), encode sub-picture identifications (IDs) of the sub-pictures into the bitstream, and encode coding blocks of the sub-pictures into the bitstream; Encoder including.

17. The encoder of claim 16, wherein the encoder is further configured to perform a method according to any one of claims 7 to 9.

18. A computer program comprising a program code for performing the method of any one of claims 1 to 5 when the computer program is run on a computer or processor.

19. A computer program comprising program code for carrying out a method according to any one of claims 6 to 9 when the computer program is run on a computer or processor.

20. 1. A device for storing and decoding a video bitstream, comprising: a communication interface, a processor, and a storage medium, the communication interface configured to receive and / or transmit a bitstream, the storage medium configured to store the bitstream, the bitstream including coded data of a picture including a subpicture, a width of the subpicture in units of coding tree blocks (CTBs), a height of the subpicture in units of CTBs, and a subpicture identification (ID) of the subpicture; The device, wherein the processor parses the bitstream to obtain the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture, and the processor further decodes coding blocks of the subpicture based on the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture.

Images