Encoder, decoder, and corresponding method

Abstract

To provide a video coding mechanism.SOLUTION: A mechanism includes receiving a bit stream including a picture including a sub-picture. The width of the sub-picture of a unit of a coating tree block (CTB) and the height of the sub-picture of the unit of the CTB are acquired from a bit stream. The coding block of the sub-picture is decoded on the basis of the width of the sub-picture and the height of the sub-picture. The coding block of the sub-picture is transferred so as to be displayed as a part of a decoding video sequence.SELECTED DRAWING: Figure 5D

Description

Technical Field

[0001] [Technical Field] The present disclosure generally relates to video coding, and more specifically to coding sub-pictures of a picture in video coding.

Background Art

[0002] The amount of video data required to depict even relatively short videos can be substantial, which can pose difficulties when the data is being streamed or communicated across a communication network with limited bandwidth capacity. Thus, video data is generally compressed before being communicated across modern telecommunications networks. Since memory resources can be limited, the size of a video can also be a concern when the video is stored on a storage device. Video compression devices often use software and / or hardware at the source to code the video data before transmission or storage, thereby reducing the amount of data required to represent the digital video image. The compressed data is then received at the destination by a video decompression device that decodes the video data. Due to limited network resources and the ever-increasing demands for higher video quality, improved compression and decompression techniques that improve the compression ratio without sacrificing much or any of the image quality are desirable.

Summary of the Invention

[0003] In embodiments, the disclosure includes a method implemented in a decoder, the method comprising: receiving a bitstream containing a picture containing subpictures with the decoder's receiver; obtaining the width and height of the subpictures in units of coding tree blocks (CTBs) from the bitstream with the decoder's processor; and decoding the coding blocks of the subpictures based on the width and height of the subpictures with the processor. A video coding system may divide a picture into subpictures. This allows different subpictures to be treated differently when coding video. For example, subpictures can be extracted and displayed separately and resized independently based on application-level changes. In some cases, subpictures may be created by dividing a picture into tiles and assigning the tiles to subpictures. Some video coding systems describe subpicture boundaries with respect to the tiles contained in the subpictures. However, the tiling scheme may not be used for some pictures. Thus, such boundary descriptions may restrict the use of subpictures to pictures that use tiles. This disclosure includes a mechanism for signaling subpicture boundaries with respect to the CTB and / or CTU. Specifically, the width and height of a subpicture can be signaled in units of the CTB. The position of the top-left CTU of the subpicture can also be signaled as an offset from the top-left CTU of the picture, as measured in the CTB. The CTU and CTB sizes may be set to predetermined values. Thus, signaling the subpicture dimensions and position with respect to the CTB and CTU provides the decoder with sufficient information to position the subpicture for display. This allows the subpicture to be used even when tiles are not being used. Furthermore, this signaling mechanism avoids complexity and can be coded using a relatively small number of bits. Thus, this example provides further functionality to the video codec by enabling the subpicture to be used independently of tiles.Furthermore, this example increases coding efficiency and, therefore, reduces the use of processor, memory, and / or network resources in the encoder and / or decoder.

[0004] Optionally, in any of the above embodiments, another implementation of that embodiment further includes the steps of: the processor obtaining the offset of a subpicture in units of CTB from a bitstream; and the processor positioning the subpicture relative to the picture based on the subpicture offset.

[0005] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the offset of the subpicture is specified as the vertical position of the upper-left coding tree unit (CTU) of the subpicture and the horizontal position of the upper-left CTU of the subpicture.

[0006] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the offset of the subpicture is further specified as the difference between the top-left CTU of the picture and the top-left CTU of the subpicture.

[0007] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the vertical position of the top-left CTU of the subpicture is stored in the bitstream as subpic_ctu_top_left_y, and the vertical position of the top-left CTU of the subpicture is stored in the bitstream as subpic_ctu_top_left_x.

[0008] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the width of the subpicture is stored in the bitstream as subpic_width_minus1 and the height of the subpicture is stored in the bitstream as subpic_height_minus1.

[0009] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the width, height, and offset of the subpicture are obtained from a sequence parameter set (SPS) in the bitstream.

[0010] In embodiments, the disclosure includes a method implemented in an encoder, the method comprising: the steps of dividing a picture into subpictures by the encoder's processor; determining the width and height of the subpictures by the processor; encoding the width and height of the subpictures into a bitstream in units of CTBs by the processor; encoding the coding blocks of the subpictures into a bitstream by the processor; and storing the bitstream in a memory coupled to the processor for communication to a decoder. A video coding system may divide a picture into subpictures. This allows different subpictures to be treated differently when coding video. For example, subpictures can be extracted and displayed separately and resized independently based on application-level changes. In some cases, subpictures may be created by dividing a picture into tiles and assigning the tiles to subpictures. Some video coding systems describe subpicture boundaries with respect to the tiles contained in the subpictures. However, the tiling scheme may not be used for some pictures. Therefore, such boundary descriptions may restrict the use of subpictures to pictures that use tiles. This disclosure includes a mechanism for signaling subpicture boundaries with respect to the CTB and / or CTU. Specifically, the width and height of the subpicture can be signaled in units of the CTB. The position of the top-left CTU of the subpicture can also be signaled as an offset from the top-left CTU of the picture, as measured in the CTB. The CTU and CTB sizes may be set to predetermined values. Thus, signaling the subpicture dimensions and position with respect to the CTB and CTU provides the decoder with sufficient information to position the subpicture for display. This allows the subpicture to be used even when the tile is not in use. Furthermore, this signaling mechanism avoids complexity and can be coded using a relatively small number of bits.Therefore, this example provides further functionality to the video codec by enabling subpictures to be used independently of tiles. Furthermore, this example increases coding efficiency and thus reduces the use of processor, memory, and / or network resources in the encoder and / or decoder.

[0011] Optionally, in any of the above embodiments, another implementation of that embodiment further includes the steps of: the processor determining the subpicture offset in units of CTB; and the processor encoding the subpicture offset in units of CTB into a bitstream.

[0012] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the offset of the subpicture is specified as the vertical position of the upper-left CTU of the subpicture and the horizontal position of the upper-left CTU of the subpicture.

[0013] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the offset of the subpicture is further specified as the difference between the top-left CTU of the picture and the top-left CTU of the subpicture.

[0014] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the vertical position of the top-left CTU of the subpicture is encoded in the bitstream as subpic_ctu_top_left_y, and the vertical position of the top-left CTU of the subpicture is encoded in the bitstream as subpic_ctu_top_left_x.

[0015] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the width of the subpicture is encoded in the bitstream as subpic_width_minus1 and the height of the subpicture is encoded in the bitstream as subpic_height_minus1.

[0016] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the width, height, and offset of the subpicture are encoded into a bitstream in the SPS.

[0017] In embodiments, the disclosure includes a video coding device comprising a processor, a receiver coupled to the processor, a memory coupled to the processor, and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform any of the methods described above.

[0018] In embodiments, the disclosure includes a non-temporary computer-readable medium containing a computer program product for use by a video coding device, the computer program product containing computer-executable instructions stored on the non-temporary computer-readable medium to cause the video coding device to perform any of the methods described above when executed by a processor.

[0019] In embodiments, the present disclosure includes a decoder comprising: receiving means for receiving a bitstream containing a picture containing subpictures; acquiring means for obtaining the width and height of the subpictures in units of CTB from the bitstream; decoding means for decoding coding blocks of the subpictures based on the width and height of the subpictures; and transferring means for transferring the coding blocks of the subpictures for display as part of a decoded video sequence.

[0020] Optionally, in any of the above embodiments, another implementation of that embodiment provides that the decoder is further configured to perform any of the above embodiments.

[0021] In an embodiment, the present disclosure includes an encoder comprising: partitioning means for partitioning a picture into sub-pictures; determination means for determining the width and height of a sub-picture; encoding means for encoding the width and height of the sub-picture in units of CTB into a bitstream and encoding a coding block of the sub-picture into the bitstream; and storage means for storing the bitstream for communication to a decoder.

[0022] Optionally, in any of the above aspects, another implementation of the aspect provides that the encoder is further configured to execute any of the methods of the above aspects.

[0023] For the purpose of clarity, any one of the above embodiments may be combined with any one or more of the other above embodiments to create new embodiments within the scope of the present disclosure.

[0024] These and other features will be more clearly understood from the following detailed description considered in conjunction with the accompanying drawings and the claims.

Brief Description of the Drawings

[0025] For a more complete understanding of the present disclosure, reference is now made to the following brief description, considered in conjunction with the accompanying drawings and detailed description, in which like reference numerals represent like parts. [Figure 1] It is a flowchart of an exemplary method for coding a video signal. [Figure 2] It is a schematic diagram of an exemplary coding and decoding (codec) system for video coding. [Figure 3] It is a schematic diagram showing an exemplary video encoder. [Figure 4] It is a schematic diagram showing an exemplary video decoder. [Figure 5A] It is a schematic diagram showing an exemplary picture partitioned into sub-pictures. [Figure 5B]It is a schematic diagram showing an exemplary subpicture divided into slices. [Figure 5C] It is a schematic diagram showing an exemplary slice divided into tiles. [Figure 5D] It is a schematic diagram showing an exemplary slice divided into coding tree units (CTUs). [Figure 6] It is a schematic diagram showing an example of one - direction inter - prediction. [Figure 7] It is a schematic diagram showing an example of bi - direction inter - prediction. [Figure 8] It is a schematic diagram showing an example of coding a current block based on candidate motion vectors from adjacent coded blocks. [Figure 9] It is a schematic diagram showing an exemplary pattern for determining a candidate list of motion vectors. [Figure 10] It is a block diagram showing an exemplary in - loop filter. [Figure 11] It is a schematic diagram showing an exemplary bitstream including coding tool parameters for supporting decoding of subpictures of a picture. [Figure 12] It is a schematic diagram of an exemplary video coding device. [Figure 13] It is a flowchart of an exemplary method for encoding a video sequence including subpictures into a bitstream. [Figure 14] It is a flowchart of an exemplary method for decoding a video sequence including subpictures from a bitstream. [Figure 15] It is a schematic diagram of an exemplary system for coding a video sequence of an image including subpictures in a bitstream.

Best Mode for Carrying Out the Invention

[0026] First, while exemplary implementations of one or more embodiments are provided below, it should be understood that the systems and / or methods of the disclosure may be implemented using any number of techniques, whether currently known or existing. This disclosure should not be limited in any way to the exemplary implementations, drawings and techniques shown below, including the exemplary designs and implementations described herein, and may be modified within the scope of the appended claims, along with the entire scope of these equivalents.

[0027] The following abbreviations are used: Adaptive Loop Filter (ALF), Coding Tree Block (CTB), Coding Tree Unit (CTU), Coding Unit (CU), Coded Video Sequence (CVS), Joint Video Experts Team (JVET), Motion-Constrained Tile Set (MCTS), Maximum Transfer Unit (MTU), Network Abstraction Layer (NAL), Picture Order Count (POC), Raw Byte Sequence Payload (RBSP), Sample Adaptive Offset (SAO), Sequence Parameter Set (SPS), Temporal Motion Vector Prediction (TMVP), Versatile Video Coding (VVC), and Working Draft. WD) is used here.

[0028] Many video compression techniques can be used to reduce the size of video files with minimal data loss. For example, video compression techniques may include performing spatial (e.g., intra-picture) prediction and / or temporal (e.g., inter-picture) prediction to reduce or eliminate data redundancy in a video sequence. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be divided into video blocks, which may also be called tree blocks, coding tree blocks (CTBs), coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are coded using spatial prediction with respect to a reference sample in an adjacent block within the same picture. Video blocks in an intercoded (P) or bidirectional (B) slice of a picture may be coded using spatial prediction with respect to a reference sample in an adjacent block within the same picture, or temporal prediction with respect to a reference sample in another reference picture. A picture may also be called a frame and / or image, and a reference picture may be called a reference frame and / or reference image. Spatial or temporal predictions produce predicted blocks representing image blocks. Residual data represents the pixel difference between the original image blocks and the predicted blocks. Thus, the intercoded blocks are encoded according to motion vectors pointing to the blocks of reference samples that form the predicted blocks, and residual data indicating the difference between the coded blocks and the predicted blocks. Intracoded blocks are encoded according to the intracoded mode and residual data. For further compression, the residual data may be transformed from pixel domains to transformation domains. These produce residual transformation coefficients, which may be quantized. The quantized transformation coefficients may first be placed in a two-dimensional array. The quantized transformation coefficients may be scanned to generate a one-dimensional vector of transformation coefficients.Entropy coding may be applied to achieve even greater compression. Such video compression techniques are described in more detail below.

[0029] To ensure that encoded video can be accurately decoded, video is encoded and decoded according to the corresponding video coding standard. Video coding standards include H.261 of the International Telecommunication Union (ITU) Standardization Sector (ITU-T), Motion Picture Experts Group (MPEG)-1 Part 2 of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC), Advanced Video Coding (AVC), also known as ITU-T's H.262 or ISO / IEC's MPEG-2 Part 2, ITU-T's H.263, ISO / IEC's MPEG-4 Part 2, ITU-T's H.264 or ISO / IEC's MPEG-4 Part 10, and High Efficiency Video Coding (HEVC), also known as ITU-T's H.265 or MPEG-H Part 2. AVC includes extensions such as Scalable Video Coding (SVC), Multiview Video Coding (MVC), Multiview Video Coding plus Depth (MVC+D), and three-dimensional (3D) AVC (3D-AVC). HEVC includes extensions such as Scalable HEVC (SHVC), Multiview HEVC (MV-HEVC), and 3D HEVC (3D-HEVC).The ITU-T and ISO / IEC Joint Video Experts Team (JVET) has begun developing a video coding standard called Versatile Video Coding (VVC). VVC is included in a Working Draft (WD), which includes JVET-M1001-v6, providing algorithm descriptions, encoder-side descriptions of the VVC WD, and reference software.

[0030] To code a video image, the image is first segmented, and the segmentation is coded into a bitstream. Various picture segmentation methods are available. For example, an image can be segmented into normal slices, dependent slices, tiles, and / or according to wavefront parallel processing (WPP). For simplicity, when HEVC segments slices into groups of CTBs for video coding, the encoder restricts the use to normal slices, dependent slices, tiles, WPP, and combinations thereof. Such segmentation can be applied to support Maximum Transfer Unit (MTU) size matching, parallel processing, and reduced end-to-end latency. The MTU indicates the maximum amount of data that can be transmitted in a single packet. If the packet payload exceeds the MTU, the payload is split into two packets through a process called fragmentation.

[0031] A normal slice, also simply called a slice, is a segmented portion of an image that, due to loop filtering behavior, can be reconstructed independently of other normal slices within the same picture, despite some interdependencies. Each normal slice is encapsulated in its own Network Abstraction Layer (NAL) unit for transmission. Furthermore, intra-picture predictions (intra-sample predictions, motion information predictions, coding mode predictions) and entropy coding dependencies across slice boundaries may be disabled to support independent reconstruction. Such independent reconstruction supports parallelization. For example, normal slice-based parallelization uses minimal inter-processor or inter-core communication. However, since each normal slice is independent, each slice is associated with an individual slice header. The use of normal slices can result in substantial coding overhead due to the bit cost of the slice header for each slice and the lack of predictions across slice boundaries. Additionally, normal slices may be used to support matching for MTU size requirements. Specifically, since a slice can normally be encapsulated in individual NAL units and coded independently, each slice should be smaller than the MTU in the MTU scheme to avoid splitting the slice into multiple packets. Therefore, the goals of parallelization and MTU size matching can impose conflicting requirements on the slice layout within the picture.

[0032] Dependent slices are similar to normal slices but have a shortened slice header and allow for the segmentation of image tree block boundaries without breaking in-picture predictions. Thus, dependent slices allow normal slices to be fragmented into multiple NAL units, which provides reduced end-to-end delay by allowing parts of the normal slice to be sent out before the encoding of the entire normal slice is complete.

[0033] A tile is a segmented portion of an image created by horizontal and vertical boundaries that make up the columns and rows of tiles. Tiles may be coded in raster scan order (right to left and top to bottom). The scan order of CTBs is local within a tile. Therefore, the CTBs in the first tile are coded in raster scan order before proceeding to the CTBs in the next tile. As with regular slices, tiles break in-picture prediction dependency and entropy decoding dependency. However, tiles do not have to be contained within individual NAL units, and therefore tiles do not have to be used for MTU size matching. Each tile may be processed by one processor / core, and inter-processor / inter-core communication used for in-picture prediction between processing units decoding adjacent tiles may be limited to passing a shared slice header (when adjacent tiles are in the same slice) and performing loop filtering-related sharing of reconstructed samples and metadata. When a slice contains more than one tile, the entry point byte offset for each tile, other than the first entry point offset in the slice, may be signaled in the slice header. For each slice and tile, at least one of the following conditions must be met: 1) all coded tree blocks within a slice belong to the same tile, and 2) all coded tree blocks within a tile belong to the same slice.

[0034] In WPP, an image is divided into single rows of CTBs. The entropy decoding and prediction mechanism may use data from CTBs in other rows. Parallel processing is possible through parallel decoding of CTB rows. For example, the current row may be decoded in parallel with the previous row. However, the decoding of the current row is delayed by two CTBs from the decoding process of the previous row. This delay ensures that data related to the CTB above and to the upper right of the current CTB in the current row is available before the current CTB is coded. This technique appears as a wavefront when represented graphically. This staggered start allows for parallelization on up to the same number of processors / cores as the number of CTB rows the image contains. Since intra-picture prediction between adjacent tree block rows within a picture is allowed, inter-processor / inter-core communication to enable intra-picture prediction can be substantial. WPP division takes NAL unit size into consideration. Therefore, WPP does not support MTU size matching. However, to achieve MTU size matching as desired, slices can usually be used with respect to WPP with a certain coding overhead.

[0035] Tiles may also include motion-constrained tile sets. A motion-constrained tile set (MCTS) is a tile set designed such that the associated motion vectors are constrained to all sample positions within the MCTS and fractional sample positions that require only all sample positions within the MCTS for interpolation. Furthermore, the use of motion vector candidates for time motion vector prediction derived from blocks outside the MCTS is prohibited. Thus, each MCTS may be decoded independently, without the presence of tiles not included in the MCTS. Supplemental enhancement information (SEI) messages for time MCTS may indicate the presence of MCTS in the bitstream and may be used to signal MCTS. The SEI messages for MCTS provide supplemental information that can be used in MCTS sub-bitstream extraction (specified as part of the semantics of the SEI message) to generate a fitted bitstream for the MCTS set. The information includes numerous extraction information sets, each defining numerous MCTS sets, and includes raw bytes sequence payload (RBSP) bytes for replacement video parameter sets (VPS), sequence parameter sets (SPS), and picture parameter sets (PPS) to be used during the MCTS sub-bitstream extraction process. When extracting sub-bitstreams according to the MCTS sub-bitstream extraction process, the parameter sets (VPS, SPS, and PPS) may be rewritten or replaced, and one or all of the slice address-related syntax elements (including first_slice_segment_in_pic_flag and slice_segment_address) may use different values ​​in the extracted sub-bitstream, so the slice header may be updated.

[0036] A picture may also be divided into one or more subpictures. Dividing a picture into subpictures can allow different parts of the picture to be treated differently from a coding perspective. For example, a subpicture can be extracted and displayed without extracting other subpictures. As another example, different subpictures can be displayed at different resolutions, rearranged relative to each other (for example, in a video conferencing application), or coded as separate pictures even if the subpictures combine data from a common picture.

[0037] An exemplary implementation of a subpicture is as follows: A picture can be divided into one or more subpictures. A subpicture is a set of rectangles or squares in slice / tile groups, starting with a slice / tile group having an address equal to 0. Each subpicture may refer to a different PPS, and therefore each subpicture may use a different division mechanism. A subpicture may be treated like a picture in the decoding process. The current reference picture used to decode the current subpicture may be generated by extracting a region from the reference picture in the decoding picture buffer that is identical in position to the current subpicture. The extracted region may also be the decoded subpicture, and therefore interpretation may be performed between subpictures of the same size and position in the picture. A tile group may also be a sequence of tiles in the tile raster scan of the subpicture. The following may be derived to determine the position of a subpicture in the picture: Each subpicture may be contained in the next unoccupied position in the CTU raster scan order in the picture, which is large enough to fit the subpicture within the picture boundary.

[0038] The subpicture schemes used by various video coding systems involve various problems that reduce coding efficiency and / or functionality. This disclosure includes various solutions to such problems. In a first exemplary problem, interpretation may be performed according to one of several interpretation modes. A particular interpretation mode generates a candidate list of motion vector predictors in both the encoder and the decoder. This allows the encoder to signal the motion vector by signaling an index from the candidate list instead of signaling the entire motion vector. Furthermore, some systems encode subpictures for independent extraction. This allows the current subpicture to be decoded and displayed without decoding information from other subpictures. This can lead to errors when motion vectors pointing outside of a subpicture are used. This is because the data pointed to by the motion vector may not be decoded and therefore may not be available.

[0039] Therefore, in the first example, a flag is disclosed here that indicates that the subpicture should be treated as a picture. This flag is set to support separate extraction of the subpicture. When the flag is set, motion vector predictors obtained from co-located blocks will only include motion vectors pointing within the subpicture. Any motion vector predictors pointing outside the subpicture are excluded. This ensures that motion vectors pointing outside the subpicture are not selected and that associated errors are avoided. Co-located blocks are blocks from a different picture than the current picture. Motion vector predictors from blocks within the current picture (non-co-located blocks) may point outside the subpicture, as other processes, such as interpolation filters, can prevent errors for such motion vector predictors. Thus, this example provides further functionality to the video encoder / decoder (codec) by preventing errors when performing subpicture extraction.

[0040] In the second example, a flag is disclosed that indicates a subpicture should be treated as a picture. When the current subpicture is treated as a picture, it should be extracted without referencing other subpictures. Specifically, this example uses a clipping function that is applied when applying an interpolation filter. This clipping function ensures that the interpolation filter does not depend on data from adjacent subpictures in order to maintain separation between subpictures to support separate extractions. Thus, the clipping function is applied when the flag is set and the motion vector points outside the current subpicture. The interpolation filter is then applied to the result of the clipping function. Thus, this example provides further functionality to the video codec by preventing errors when performing subpicture extraction. Thus, the first and second examples address the first exemplary problem.

[0041] In the second exemplary problem, the video coding system divides a picture into subpictures, slices, tiles, and / or coding tree units, which are then divided into blocks. Such blocks are then encoded for transmission to a decoder. Decoding such blocks may result in a decoded image containing various types of noise. To correct such problems, the video coding system may apply various filters across block boundaries. These filters can remove blocking, quantization noise, and other undesirable coding artifacts. As described above, some systems encode subpictures for independent extraction. This allows the current subpicture to be decoded and displayed without decoding information from other subpictures. In such systems, subpictures may be divided into blocks for encoding. Thus, block boundaries along subpicture edges may be aligned with subpicture boundaries. In some cases, block boundaries may also be aligned with tile boundaries. Filters may be applied across such block boundaries and therefore across subpicture boundaries and / or tile boundaries. This can cause errors when the current subpicture is extracted independently, as the filtering process may behave in unexpected ways when data from adjacent subpictures is unavailable.

[0042] In the third example, a flag is disclosed that controls filtering at the sub-picture level. When the flag is set for a sub-picture, the filter can be applied across sub-picture boundaries. When the flag is not set, the filter is not applied across sub-picture boundaries. Thus, the filter can be turned off for sub-pictures encoded for separate extraction, or turned on for sub-pictures encoded for display as a group. Therefore, this example provides further functionality to the video codec by preventing filter-related errors when performing sub-picture extraction.

[0043] In the fourth example, a flag is disclosed that can be set to control filtering at the tile level. When the flag is set for a tile, the filter can be applied across tile boundaries. When the flag is not set, the filter is not applied across tile boundaries. Thus, the filter can be turned off or on for use at tile boundaries (for example, while continuing to filter within the tile). Therefore, this example provides further functionality to the video codec by supporting selective filtering across tile boundaries. Thus, the third and fourth examples address the second exemplary problem.

[0044] In a third exemplary problem, a video coding system may divide a picture into subpictures. This allows different subpictures to be treated differently when coding video. For example, subpictures can be extracted and displayed separately and resized independently based on application-level changes. In some cases, subpictures may be created by dividing a picture into tiles and assigning the tiles to the subpictures. Some video coding systems describe subpicture boundaries with respect to the tiles contained within the subpictures. However, the tiling method may not be used for some pictures. Therefore, such boundary descriptions may restrict the use of subpictures to pictures that use tiles.

[0045] In the fifth example, a mechanism for signaling subpicture boundaries with respect to the CTB and / or CTU is disclosed. Specifically, the width and height of the subpicture can be signaled in units of the CTB. The position of the top-left CTU of the subpicture can also be signaled as an offset from the top-left CTU of the picture, as measured in the CTB. The CTU and CTB sizes may be set to predetermined values. Thus, signaling the subpicture dimensions and position with respect to the CTB and CTU provides the decoder with sufficient information to position the subpicture for display. This allows the subpicture to be used even when the tile is not being used. Furthermore, this signaling mechanism avoids complexity and can be coded using a relatively small number of bits. Thus, this example provides further functionality to the video codec by allowing the subpicture to be used independently of the tile. In addition, this example increases coding efficiency and therefore reduces the use of processor, memory and / or network resources in the encoder and / or decoder. Thus, the fifth example addresses the third exemplary problem.

[0046] In the fourth exemplary problem, a picture can be divided into multiple slices for encoding. In some video coding systems, slices are addressed based on their positions relative to the picture. Still other video coding systems use the concept of subpictures. As mentioned above, subpictures can be treated differently from other subpictures from a coding perspective. For example, a subpicture can be extracted and displayed independently of other subpictures. In such cases, the slice addresses generated based on the picture positions may not work properly because a considerable number of assumed slice addresses are omitted. Some video coding systems address this problem by dynamically rewriting the slice header upon request to change the slice address in order to support subpicture extraction. Such a process can be resource-intensive because it may occur every time the user requests to view a subpicture.

[0047] In the sixth example, a slice addressed to a subpicture containing slices is disclosed. For example, the slice header may include a subpicture identifier (ID) and the addresses of each slice contained within the subpicture. Furthermore, the sequence parameter set (SPS) may include the dimensions of the subpicture that can be referenced by the subpicture ID. Thus, when separate extraction of subpictures is required, the slice header does not need to be rewritten. The slice header and SPS contain sufficient information to support the positioning of slices within the subpicture for display. Thus, this example increases coding efficiency and / or avoids redundant rewriting of the slice header, and therefore reduces the use of processor, memory, and / or network resources in the encoder and / or decoder. Thus, the sixth example addresses the fourth exemplary problem.

[0048] Figure 1 is a flowchart of an exemplary operation method 100 for coding a video signal. Specifically, the video signal is encoded in an encoder. The encoding process compresses the video signal by using various mechanisms to reduce the video file size. A smaller file size allows the compressed video file to be transmitted to the user, while reducing the associated bandwidth overhead. The decoder then decodes the compressed video file to reconstruct the original video signal for display to the end user. The decoding process generally mirrors the encoding process to enable the decoder to consistently reconstruct the video signal.

[0049] In step 101, a video signal is input to the encoder. For example, the video signal may be an uncompressed video file stored in memory. In another example, the video file may be captured by a video capture device such as a video camera and encoded to support live streaming of the video. The video file may contain both audio and video components. The video component contains a series of image frames that give a visual impression of motion when viewed in sequence. Each frame contains pixels, which are represented with respect to light and here are called lumina components (or lumina samples), and colors, which are called chroma components (or color samples). In some examples, the frames may also contain depth values ​​to support three-dimensional display.

[0050] In step 103, the video is divided into blocks. The division involves subdividing the pixels within each frame into square and / or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H Part 2), a frame can first be divided into coding tree units (CTUs), which are blocks of a predetermined size (e.g., 64 pixels × 64 pixels). A CTU contains both luminous and chroma samples. The coding tree may be used to divide the CTUs into blocks, and then to recursively subdivide the blocks until a configuration supporting further coding is achieved. For example, the luminous component of a frame may be subdivided until the individual blocks contain relatively uniform illumination values. Furthermore, the chroma component of a frame may be subdivided until the individual blocks contain relatively uniform color values. Thus, the division mechanism varies depending on the content of the video frame.

[0051] In step 105, various compression mechanisms are used to compress the image blocks divided in step 103. For example, inter-prediction and / or intra-prediction may be used. Inter-prediction is designed to take advantage of the fact that objects in a common scene tend to appear in consecutive frames. Therefore, a block representing an object in a reference frame does not need to be described repeatedly in adjacent frames. Specifically, an object such as a table may remain in the same position across multiple frames. Thus, a table is described once, and adjacent frames can refer back to the reference frame. Pattern matching mechanisms may be used to match objects across multiple frames. Furthermore, moving objects may be represented across multiple frames, for example, due to the movement of the object or the movement of the camera. As a particular example, a video may show a car moving across the screen across multiple frames. Motion vectors can be used to describe such motion. A motion vector is a two-dimensional vector that provides an offset from the coordinates of an object in a frame to the coordinates of an object in a reference frame. Therefore, inter-prediction can encode an image block in the current frame as a set of motion vectors that represent the offset from the corresponding block in the reference frame.

[0052] Intra-prediction encodes blocks within a common frame. It leverages the fact that luma and chroma components tend to cluster within a frame. For example, green fragments in a part of a tree tend to be adjacent to similar green fragments. Intra-prediction uses multiple directional prediction modes (e.g., 33 in HEVC), planar mode, and direct current (DC) mode. Directional modes indicate that the current block is similar to / identical to samples of adjacent blocks in the corresponding direction. Planar mode indicates that a series of blocks along a row / column (e.g., a plane) can be interpolated based on adjacent blocks at the edges of the row. Planar mode effectively shows smooth light / color transitions across rows / columns by using a relatively constant slope when changing values. DC mode is used for boundary smoothing, indicating that a block is similar to / identical to the mean value related to samples of all adjacent blocks related to the angular direction of the directional prediction mode. Thus, intra-predicted blocks can represent image blocks as various relational prediction mode values ​​instead of actual values. Furthermore, the interpretation block can represent the image block as a motion vector value instead of its actual value. In either case, the prediction block may not accurately represent the image block in some cases. Any difference is stored in the residual block. To further compress the file, a transformation may be applied to the residual block.

[0053] In step 107, various filtering techniques may be applied. In HEVC, filters are applied according to an in-loop filtering scheme. The block-based prediction described above may result in the generation of blocky images in the decoder. Furthermore, the block-based prediction scheme may reconstruct the encoded blocks for later use as reference blocks after encoding them. The in-loop filtering scheme repeatedly applies noise suppression filters, deblocking filters, adaptive loop filters, and sample adaptive offset (SAO) filters to blocks / frames. These filters mitigate such blocking artifacts so that the encoded file can be accurately reconstructed. Furthermore, these filters mitigate artifacts in the reconstructed reference blocks so that the artifacts are less likely to create further artifacts in subsequent blocks encoded based on the reconstructed reference blocks.

[0054] Once the video signal has been segmented, compressed, and filtered, in step 109 the resulting data is encoded into a bitstream. The bitstream includes the above data and any signaling data that is desirable to support the proper reproduction of the video signal in the decoder. For example, such data may include segmentation data, prediction data, residual blocks, and various flags that provide coding instructions to the decoder. The bitstream may be stored in memory for transmission to the decoder on request. The bitstream may also be broadcast and / or multicast to multiple decoders. The creation of the bitstream is an iterative process. Therefore, steps 101, 103, 105, 107, and 109 may occur sequentially and / or simultaneously across many frames and blocks. The order shown in Figure 1 is presented for clarity and ease of explanation and is not intended to limit the video coding process to a specific order.

[0055] The decoder receives the bitstream and begins the decoding process in step 111. Specifically, the decoder uses an entropy decoding scheme to convert the bitstream into corresponding syntax and video data. In step 111, the decoder uses the syntax data from the bitstream to determine the frame divisions. The divisions should match the result of the block division in step 103. Herein lies the entropy coding / decoding used in step 111. The encoder makes many choices during the compression process, such as selecting a block division scheme from several possible options based on the spatial positioning of values ​​in the input image. Signaling the exact choice may involve using a number of bins. When used here, a bin is a binary value (e.g., a bit value that can change depending on the context) treated as a variable. Entropy coding allows the encoder to discard any option that is obviously unfeasible in particular, leaving a set of acceptable options. Each acceptable option is then assigned a codeword. The length of the codeword is based on the number of acceptable options (e.g., one bin for two options, two bins for three or four options, etc.). Next, the encoder encodes a codeword for the selected option. This method reduces the size of the codeword, as it is desirable that the codeword be large enough to uniquely indicate a selection from a small subset of acceptable options, as opposed to uniquely indicating a selection from a potentially large set of all possible options. The decoder then decodes the selection by determining the set of acceptable options in a similar manner to the encoder. By determining the set of acceptable options, the decoder can read the codeword and determine the selection made by the encoder.

[0056] In step 113, the decoder performs block decoding. Specifically, the decoder uses an inverse transform to generate residual blocks. The decoder then uses the residual blocks and corresponding prediction blocks to reconstruct the image blocks according to the segmentation. The prediction blocks may include both intra-prediction blocks and inter-prediction blocks, as generated by the encoder in step 105. The reconstructed image blocks are then positioned in the frames of the reconstructed video signal according to the segmentation data determined in step 111. The syntax for step 113 may also be signaled in the bitstream via entropy coding as described above.

[0057] In step 115, filtering is performed on the frames of the reconstructed video signal in a manner similar to that in step 107 of the encoder. For example, noise suppression filters, deblocking filters, adaptive loop filters, and SAO filters may be applied to the frames to remove blocking artifacts. Once the frames have been filtered, the video signal can be output to a display in step 117 for viewing by the end user.

[0058] Figure 2 is a schematic diagram of an exemplary coding and decoding (codec) system 200 for video coding. Specifically, the codec system 200 provides functions to support the implementation of operation method 100. The codec system 200 is generalized to show components used in both the encoder and decoder. The codec system 200 receives and segments the video signal as described in relation to steps 101 and 103 of operation method 100, which produces segmented video signals 201. The codec system 200 then compresses the segmented video signals 201 into a coded bitstream when functioning as an encoder, as described in relation to steps 105, 107 and 109 of method 100. When functioning as a decoder, the codec system 200 generates an output video signal from the bitstream, as described in relation to steps 111, 113, 115 and 117 of operation method 100. The codec system 200 includes an overall coder control component 211, a transform scaling and quantization component 213, an intra-picture estimation component 215, an intra-picture prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a filter control analysis component 227, an in-loop filter component 225, a decoded picture buffer component 223, and a header format and context adaptive binary arithmetic coding (CABAC) component 231. These components are combined as shown in the figure. In Figure 2, black lines indicate the movement of data to be encoded / decoded, and dashed lines indicate the movement of control data that controls the operation of other components. All components of the codec system 200 may also be present in the encoder. The decoder may include a subset of the components of the codec system 200.For example, the decoder may include an intra-picture prediction component 217, a motion compensation component 219, a scaling and inverse transform component 229, an in-loop filter component 225, and a decoded picture buffer component 223. These components are described below.

[0059] The segmented video signal 201 is a captured video sequence segmented into blocks of pixels by a coding tree. The coding tree uses various segmentation modes to subdivide blocks of pixels into smaller blocks of pixels. These blocks can then be further subdivided into even smaller blocks. Blocks may also be called nodes on the coding tree. Larger parent nodes are subdivided into smaller child nodes. The number of times a node is subdivided is called the node / coding tree depth. In some cases, segmented blocks can be contained within a coding unit (CU). For example, a CU may be a sub-part of a CTU containing lumens, red difference chroma (Cr) blocks, and blue difference chroma (Cb) blocks, along with corresponding syntax instructions for the CU. The segmentation modes may include binary trees (BT), triple trees (TT), and quad trees (QT), which are used to segment nodes into two, three, or four child nodes of varying shapes, depending on the segmentation mode used. The segmented video signal 201 is transferred for compression to the overall coder control component 211, the transformation scaling and quantization component 213, the intrapicture estimation component 215, the filter control analysis component 227, and the motion estimation component 221.

[0060] The overall coder control component 211 is configured to make decisions related to coding images of a video sequence into a bitstream, according to application constraints. For example, the overall coder control component 211 manages the optimization of bitrate / bitstream size for reproduction quality. Such decisions may be based on memory space / bandwidth availability and image resolution requirements. The overall coder control component 211 also manages buffer utilization, taking transmission speed into consideration, to mitigate buffer underrun and overrun issues. To manage these issues, the overall coder control component 211 manages segmentation, prediction, and filtering by other components. For example, the overall coder control component 211 may dynamically increase compression complexity to increase resolution and bandwidth usage, or decrease compression complexity to decrease resolution and bandwidth usage. Thus, the overall coder control component 211 controls other components of the codec system 200 to balance bitrate concerns with video signal reproduction quality. The overall coder control component 211 creates control data that controls the operation of other components. The control data is also transferred to the header format and CABAC component 231 to be encoded into a bitstream for signaling parameters for decoding in the decoder.

[0061] The segmented video signal 201 is also sent to the motion estimation component 221 and the motion compensation component 219 for interpretation. A frame or slice of the segmented video signal 201 may be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform interpretation coding of the received video blocks for one or more blocks within one or more reference frames in order to provide time predictions. The codec system 200 may perform multiple coding passes, for example, to select an appropriate coding mode for each block of video data.

[0062] The motion estimation component 221 and the motion compensation component 219 may be highly integrated, but are illustrated separately for conceptual purposes. The motion estimation performed by the motion estimation component 221 is the process of generating motion vectors that estimate the motion about a video block. The motion vectors may, for example, represent the displacement of an object coded with respect to a prediction block. A prediction block is a block that has been found to closely match the block to be coded with respect to pixel difference. A prediction block may also be called a reference block. Such pixel difference may be determined by the sum of absolute difference (SAD), the sum of square difference (SSD), or other difference metrics. HEVC uses several coded objects, including CTUs, coding tree blocks (CTBs), and CUs. For example, a CTU can be split into CTBs, which can then be split into CBs to be included in a CU. A CU can be coded as a prediction unit (PU) containing prediction data and / or a transform unit (TU) containing transform residual data about the CU. The motion estimation component 221 generates motion vectors, PUs, and TUs by using rate distortion analysis as part of the rate distortion optimization process. For example, the motion estimation component 221 may determine multiple reference blocks, multiple motion vectors, etc. for the current block / frame, or it may select the reference blocks, motion vectors, etc. that have the best rate distortion characteristics. The best rate distortion characteristics balance both the quality of video reproduction (e.g., the amount of data loss due to compression) and coding efficiency (e.g., the size of the final encoding).

[0063] In some examples, the codec system 200 may calculate values ​​for sub-integer pixel positions of the reference picture stored in the decoded picture buffer component 223. For example, the video codec system 200 may interpolate values ​​for quarter-pixel, eighth-pixel, or other fractional pixel positions of the reference picture. Thus, the motion estimation component 221 may perform motion searches on all pixel positions and fractional pixel positions and output motion vectors with fractional pixel precision. The motion estimation component 221 calculates motion vectors for the PU of video blocks in the intercoded slice by comparing the PU positions with the predicted block positions of the reference picture. For encoding, the motion estimation component 221 outputs the calculated motion vectors as motion data to the header format and CABAC component 231, and the motion to the motion compensation component 219.

[0064] Motion compensation performed by the motion compensation component 219 may involve fetching or generating a predicted block based on the motion vector determined by the motion estimation component 221. Similarly, in some examples, the motion estimation component 221 and the motion compensation component 219 may be functionally integrated. Upon receiving the motion vector for the PU of the current video block, the motion compensation component 219 may identify the predicted block pointed to by the motion vector. A residual video block is then formed by subtracting the pixel values ​​of the predicted block from the pixel values ​​of the coded current video block, thereby forming the pixel difference values. Generally, the motion estimation component 221 performs motion estimation on the lumens component, and the motion compensation component 219 uses the motion vector calculated based on the lumens component for both the chromens and lumens components. The predicted and residual blocks are then transferred to the transformation scaling and quantization component 213.

[0065] The segmented video signal 201 is also transmitted to the intra-picture estimation component 215 and the intra-picture prediction component 217. Similar to the motion estimation component 221 and the motion compensation component 219, the intra-picture estimation component 215 and the intra-picture prediction component 217 may be highly integrated, but are illustrated separately for conceptual purposes. The intra-picture estimation component 215 and the intra-picture prediction component 217 intra-predict the current block for the block in the current frame, instead of the inter-prediction performed by the motion estimation component 221 and the motion compensation component 219 between frames, as described above. In particular, the intra-picture estimation component 215 determines the intra-prediction mode to use for encoding the current block. In some examples, the intra-picture estimation component 215 selects an appropriate intra-prediction mode for encoding the current block from several tested intra-prediction modes. The selected intra-prediction mode is then transmitted to the header format and CABAC component 231 for encoding.

[0066] For example, the intra-picture estimation component 215 calculates rate distortion values ​​for various tested intra-prediction modes using rate distortion analysis and selects the intra-prediction mode with the best rate distortion characteristics among the tested modes. Rate distortion analysis generally determines the amount of distortion (or error) between the original unencoded blocks encoded to generate the encoded blocks and the encoded blocks, and the bit rate (e.g., number of bits) used to generate the encoded blocks. The intra-picture estimation component 215 calculates a ratio from the distortion and rate for various encoded blocks to determine which intra-prediction mode exhibits the best rate distortion value for a block. Furthermore, the intra-picture estimation component 215 may be configured to code depth blocks of a depth map using a depth modeling mode (DMM) based on rate-distortion optimization (RDO).

[0067] The intrapicture prediction component 217, when implemented in an encoder, may generate residual blocks from prediction blocks based on a selected intrapicture prediction mode determined by the intrapicture estimation component 215, or, when implemented in a decoder, may read residual blocks from the bitstream. The residual blocks contain the difference in values ​​between the prediction blocks and the original blocks, represented as a matrix. The residual blocks are then transferred to the transformation scaling and quantization component 213. The intrapicture estimation component 215 and the intrapicture prediction component 217 may be performed on both the luma and chroma components.

[0068] The transform scaling and quantization component 213 is configured to further compress the residual block. The transform scaling and quantization component 213 applies a transform such as a discrete cosine transform (DCT), discrete sine transform (DST), or a conceptually similar transform to the residual block to generate a video block containing residual transform coefficient values. Wavelet transforms, integer transforms, subband transforms, or other types of transforms may also be used. The transform may convert the residual information from a pixel value domain to a transform domain such as a frequency domain. The transform scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling involves applying a scale factor to the residual information so that different frequency information is quantized at different granularities, which may affect the final visual quality of the reproduced video. The transform scaling and quantization component 213 is also configured to quantize the transform coefficients to further reduce the bitrate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting the quantization parameters. In some examples, the transformation scaling and quantization component 213 may then perform a scan of a matrix containing the quantized transformation coefficients. The quantized transformation coefficients are then transferred to the header format and CABAC component 231 for encoding into a bitstream.

[0069] The scaling and inverse transform component 229 applies the reverse operation of the transform scaling and quantization component 213 to support motion estimation. The scaling and inverse transform component 229 applies inverse scaling, transform, and / or quantization to reconstruct the residual block in the pixel domain for later use as a reference block that may later become a predicted block for other current blocks. The motion estimation component 221 and / or motion compensation component 219 may compute the reference block by returning the residual block to the corresponding predicted block and adding it for use in motion estimation of later blocks / frames. Filters are applied to the reconstructed reference block to mitigate artifacts created during scaling, quantization, and transform. Otherwise, such artifacts could lead to inaccurate predictions (and create further artifacts) when subsequent blocks are predicted.

[0070] The filter-controlled analysis component 227 and the in-loop filter component 225 apply filters to residual blocks and / or reconstructed image blocks. For example, a transformed residual block from the scaling and inverse transform component 229 may be combined with the corresponding predicted block from the intra-picture prediction component 217 and / or the motion compensation component 219 to reconstruct the original image block. The filter may then be applied to the reconstructed image block. In some examples, the filter may be applied to the residual block instead. Like the other components in Figure 2, the filter-controlled analysis component 227 and the in-loop filter component 225 are highly integrated and may be implemented together, but are illustrated separately for conceptual purposes. A filter applied to a reconstructed reference block is applied to a specific spatial region and includes several parameters to adjust how such a filter is applied. The filter-controlled analysis component 227 analyzes the reconstructed reference block and sets the corresponding parameters to determine where such a filter should be applied. Such data is transferred to the header format and CABAC component 231 as filter-controlled data for encoding. The in-loop filter component 225 applies such filters based on filter control data. These filters may include deblocking filters, noise suppression filters, SAO filters, and adaptive loop filters. Depending on the example, such filters may be applied in the spatial / pixel domain (e.g., reconstructed pixel blocks) or the frequency domain.

[0071] When operating as an encoder, the filtered and reconstructed image blocks, residual blocks, and / or predicted blocks are stored in the decoded picture buffer component 223 for later use in motion estimation as described above. When operating as a decoder, the decoded picture buffer component 223 stores the reconstructed and filtered blocks and transfers them to the display as part of the output video signal. The decoded picture buffer component 223 may be any memory device capable of storing the predicted blocks, residual blocks, and / or reconstructed image blocks.

[0072] The header format and CABAC component 231 receive data from various components of the codec system 200 and encode such data into a coded bitstream for transmission to the decoder. Specifically, the header format and CABAC component 231 generate various headers for encoding control data such as overall control data and filter control data. Furthermore, prediction data, including intra-prediction and motion data, and residual data in the form of quantized transformation coefficient data are all encoded into a bitstream. The final bitstream contains all the information desired by the decoder to reconstruct the original segmented video signal 201. Such information may also include an intra-prediction mode index table (also called a codeword mapping table), definitions of coding contexts for various blocks, indications of the most probable intra-prediction mode, indications of segmentation information, etc. Such data may be encoded using entropy coding. For example, information may be encoded using context-adaptive variable length coding (CAVLC), CABAC, syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or other entropy coding techniques. Following entropy coding, the coded bitstream may be transmitted to another device (e.g., a video decoder) or archived for later transmission or retrieval.

[0073] Figure 3 is a block diagram illustrating an exemplary video encoder 300. The video encoder 300 may be used to implement the encoding function of the codec system 200 and / or to implement steps 101, 103, 105, 107 and / or 109 of the operation method 100. The encoder 300 divides the input video signal, producing a divided video signal 301, which is substantially similar to the divided video signal 201. The divided video signal 301 is then compressed by the components of the encoder 300 and encoded into a bitstream.

[0074] Specifically, the segmented video signal 301 is transferred to the intra-picture prediction component 317 for intra-prediction. The intra-picture prediction component 317 may be substantially the same as the intra-picture estimation component 215 and the intra-picture prediction component 217. The segmented video signal 301 is also transferred to the motion compensation component 321 for inter-prediction based on a reference block in the decoded picture buffer component 323. The motion compensation component 321 may be substantially the same as the motion estimation component 221 and the motion compensation component 219. The prediction block and residual block from the intra-picture prediction component 317 and the motion compensation component 321 are transferred to the transformation and quantization component 313 for transformation and quantization of the residual block. The transformation and quantization component 313 may be substantially the same as the transformation scaling and quantization component 213. The transformed and quantized residual block and the corresponding prediction block (along with the associated control data) are transferred to the entropy coding component 331 for coding into a bitstream. The entropy coding component 331 may be substantially the same as the header format and the CABAC component 231.

[0075] The transformed and quantized residual blocks and / or corresponding predicted blocks are also transferred from the transform and quantization component 313 to the inverse transform and quantization component 329 for reproduction as reference blocks for use by the motion compensation component 321. The inverse transform and quantization component 329 may be substantially the same as the scaling and inverse transform component 229. The in-loop filters in the in-loop filter component 325 are also applied to the residual blocks and / or reproduced reference blocks, depending on the example. The in-loop filter component 325 may be substantially the same as the filter-controlled analysis component 227 and the in-loop filter component 225. The in-loop filter component 325 may contain multiple filters, as described with respect to the in-loop filter component 225. The filtered blocks are then stored in the decoded picture buffer component 323 for use as reference blocks by the motion compensation component 321. The decoded picture buffer component 323 may be substantially the same as the decoded picture buffer component 223.

[0076] Figure 4 is a block diagram illustrating an exemplary video decoder 400. The video decoder 400 may be used to implement the decoding function of the codec system 200 and / or to implement steps 111, 113, 115 and / or 117 of the operation method 100. The decoder 400, for example, receives a bitstream from the encoder 300 and generates an output video signal reconstructed based on the bitstream for display to the end user.

[0077] The bitstream is received by the entropy decoding component 433. The entropy decoding component 433 is configured to implement an entropy decoding scheme such as CAVLC, CABAC, SBAC, PIPE coding, or other entropy coding techniques. For example, the entropy decoding component 433 may use header information to provide context for interpreting further data encoded as codewords in the bitstream. The decoded information includes any desired information for decoding the video signal, such as overall control data, filter control data, segmentation information, motion data, prediction data, and quantized transformation coefficients from residual blocks. The quantized transformation coefficients are transferred to the inverse transform and quantization component 429 for reconstruction into residual blocks. The inverse transform and quantization component 429 may be similar to the inverse transform and quantization component 329.

[0078] The reproduced residual blocks and / or predicted blocks are transferred to the intra-picture prediction component 417 to reproduce them as image blocks based on the intra-prediction operation. The intra-picture prediction component 417 may be the same as the intra-picture estimation component 215 and the intra-picture prediction component 217. Specifically, the intra-picture prediction component 417 uses a prediction mode to identify a reference block in the frame and applies the residual blocks to the result to reproduce the intra-predicted image block. The reproduced intra-predicted image block and / or residual block and the corresponding intra-prediction data are transferred to the decoded picture buffer component 423 via the in-loop filter component 425, which may be substantially the same as the decoded picture buffer component 223 and the in-loop filter component 225, respectively. The in-loop filter component 425 filters the reproduced image block, residual block and / or predicted block, and such information is stored in the decoded picture buffer component 423. The reconstructed image blocks from the decoded picture buffer component 423 are transferred to the motion compensation component 421 for interpretation. The motion compensation component 421 may be substantially the same as the motion estimation component 221 and / or motion compensation component 219. Specifically, the motion compensation component 421 uses motion vectors from a reference block to generate a prediction block and applies a residual block to the result to reconstruct the image block. The resulting reconstructed block may also be transferred to the decoded picture buffer component 423 via the in-loop filter component 425. The decoded picture buffer component 423 continues to store further reconstructed image blocks, which can be reconstructed into frames via piecewise information. Such frames may also be arranged in a sequence. The sequence is output to a display as a reconstructed output video signal.

[0079] Figure 5A is a schematic diagram showing an exemplary picture 500 divided into subpictures 510. For example, picture 500 can be divided for encoding by codec system 200 and / or encoder 300, and can be divided for decoding by codec system 200 and / or decoder 400. As another example, picture 500 may be divided by encoder in step 103 of method 100 for use by decoder in step 111.

[0080] Picture 500 is an image that represents the complete visual portion of a video sequence at a specified time position. Picture 500 may also be called an image and / or frame. Picture 500 may be specified by a picture order count (POC), which is an index indicating the output / display order of Picture 500 in the video sequence. Picture 500 can be divided into subpictures 510. A subpicture 510 is a rectangular or square area of ​​one or more slice / tile groups within Picture 500. Subpictures 510 are optional; therefore, some video sequences may contain subpictures 510, while others do not. Although four subpictures 510 are shown, Picture 500 can be divided into any number of subpictures 510. The division of subpictures 510 may be consistent throughout the entire coded video sequence.

[0081] Subpictures 510 may be used to allow different areas of picture 500 to be treated differently. For example, a designated subpicture 510 may be extracted independently and transmitted to the decoder. As a concrete example, a user using a virtual reality (VR) headset may see a subset of picture 500, which may give the user the impression of physically being present in the space shown in picture 500. In such a case, streaming only the subpictures 510 that may be displayed to the user may increase coding efficiency. As another example, different subpictures 510 may be treated differently in a particular application. As a concrete example, a video conferencing application may display an active speaker in a more prominent position and at a higher resolution than a user who is not currently speaking. Positioning different users within different subpictures 510 supports real-time reconstruction of the displayed image to support this functionality.

[0082] Each subpicture 510 can be identified by a unique subpicture ID, which may be consistent across the entire CVS. For example, the top-left subpicture 510 of picture 500 may have a subpicture ID of 0. In such a case, the top-left subpicture 510 of any picture 500 in the sequence can be referenced by subpicture ID 0. Furthermore, each subpicture 510 may include a defined configuration, which may be consistent across the entire CVS. For example, a subpicture 510 may include a height, width, and / or offset. The height and width describe the size of the subpicture 510, and the offset describes the position of the subpicture 510. For example, the sum of all widths of subpicture 510 in a row is the width of picture 500. Furthermore, the sum of all heights of subpicture 510 in a column is the height of picture 500. Furthermore, the offset indicates the position of the top-left corner of subpicture 510 relative to the top-left corner of picture 500. The height, width, and offset of the subpicture 510 provide sufficient information to position the corresponding subpicture 510 within the picture 500. Since the subpicture division of the subpicture 510 may be consistent across the entire CVS, the parameters associated with the subpicture may be included in the sequence parameter set (SPS).

[0083] Figure 5B is a schematic diagram showing an exemplary subpicture 510 divided into slices 515. As shown, a subpicture 510 of picture 500 may contain one or more slices 515. A slice 515 is an integer number of complete tiles or an integer number of consecutive complete CTU rows within a picture tile that is exclusively contained in a single network abstraction layer (NAL) unit. Although four slices 515 are shown, a subpicture 510 may contain any number of slices 515. A slice 515 contains visual data specific to picture 500 of a given POC. Therefore, parameters associated with a slice 515 may be contained in a picture parameter set (PPS) and / or slice header.

[0084] Figure 5C is a schematic diagram showing an exemplary slice 515 divided into tiles 517. As shown, slice 515 of picture 500 may contain one or more tiles 517. Tiles 517 may be created by dividing picture 500 into rectangular rows and columns. Thus, tile 517 is a rectangular or square area of ​​CTU within a particular tile column and a particular tile row in the picture. Tiling is optional, and therefore some video sequences will contain tiles 517, while others will not. Although four tiles 517 are shown, slice 515 may contain any number of tiles 517. Tiles 517 may contain visual data specific to slice 515 of picture 500 of a given POC. In some cases, slice 515 may also contain tiles 517. Thus, parameters related to tiles 517 may be included in the PPS and / or slice header.

[0085] Figure 5D is a schematic diagram showing an exemplary slice 515 divided into CTUs 519. As shown, slice 515 of picture 500 (or tile 517 of slice 515) may contain one or more CTUs 519. A CTU 519 is a region of picture 500 subdivided by a coding tree to create coding blocks to be encoded / decoded. A CTU 519 may contain lumens for a monochrome picture 500, or a combination of lumens and chromens for a color picture 500. A grouping of lumens or chromens that can be divided by a coding tree is called a coding tree block (CTB) 518. Thus, a CTU 519 may contain a CTB 518 of lumens and two corresponding CTBs 518 of chromens for a picture 500 having three sample sequences, or a CTB 518 of samples for a monochrome picture, or a picture coded using a syntax structure used to code three separate color planes and samples.

[0086] As shown above, picture 500 may be divided into subpictures 510, slices 515, tiles 517, CTU 519 and / or CTB 518, which are then divided into blocks. Such blocks are then encoded for transmission toward a decoder. Decoding such blocks may result in a decoded image containing various types of noise. To correct such problems, the video coding system may apply various filters across the block boundaries. These filters can remove blocking, quantization noise, and other undesirable coding artifacts. As shown above, subpicture 510 may be used when performing independent extraction. In this case, the current subpicture 510 may be decoded and displayed without decoding information from other subpictures 510. Thus, block boundaries along the edges of subpicture 510 may be aligned with subpicture boundaries. In some cases, block boundaries may also be aligned with tile boundaries. Filters may be applied across such block boundaries and therefore across subpicture boundaries and / or tile boundaries. This can cause errors when the current subpicture 510 is extracted independently, as the filtering process may behave in an unexpected way when data from the adjacent subpicture 510 is unavailable.

[0087] To address these issues, a flag may be used to control filtering at the subpicture 510 level. For example, the flag may be shown as loop_filter_across_subpic_enabled_flag. When the flag is set for a subpicture 510, the filter can be applied across the corresponding subpicture boundary. When the flag is not set, the filter is not applied across the corresponding subpicture boundary. In this way, the filter can be turned off for subpictures 510 encoded for separate extraction, or turned on for subpictures 510 encoded for display as a group. Another flag can be set to control filtering at the tile 517 level. The flag may be shown as loop_filter_across_tiles_enabled_flag. When the flag is set for a tile 517, the filter can be applied across the tile boundary. When the flag is not set, the filter is not applied across the tile boundary. In this way, the filter can be turned off or on for use at tile boundaries (for example, while continuing to filter within the tile). When used here, the filter is applied across the subpicture 510 or tile 517 boundary when the filter is applied to samples on both sides of the boundary.

[0088] Furthermore, as mentioned above, tiling is optional. However, some video coding systems describe subpicture boundaries with respect to tiles 517 contained within a subpicture 510. In such systems, the subpicture boundary description with respect to tiles 517 restricts the use of subpicture 510 to pictures 500 that use tiles 517. To expand the applicability of subpicture 510, subpicture 510 may be described with respect to boundaries with respect to CTB 518 and / or CTU 519. Specifically, the width and height of subpicture 510 can be signaled in units of CTB 518. In addition, the position of the top-left CTU 519 of subpicture 510 can be signaled as an offset from the top-left CTU 519 of picture 500 as measured in CTB 518. The sizes of CTU 519 and CTB 518 may be set to predetermined values. Therefore, signaling the subpicture dimensions and position with respect to CTB518 and CTU519 provides the decoder with sufficient information to position subpicture 510 for display. This allows subpicture 510 to be used even when tile 517 is not being used.

[0089] Furthermore, some video coding systems address slice 515 based on its position relative to picture 500. This poses a problem when subpictures 510 are coded for independent extraction and display. In such cases, slice 515 and its corresponding address associated with the omitted subpicture 510 are also omitted. The omission of the address of slice 515 can prevent the decoder from properly positioning slice 515. Some video coding systems address this problem by dynamically rewriting the address in the slice header associated with slice 515. Since the user may request any of the subpictures, such rewriting occurs every time the user requests video, which is extremely resource-intensive. To overcome this problem, slice 515 is addressed to the subpicture 510 containing slice 515 when the subpicture 510 is used. For example, slice 515 can be identified by an index or other value specific to the subpicture 510 containing slice 515. The slice address can be coded in the slice header associated with slice 515. The subpicture ID of subpicture 510, which includes slice 515, can also be encoded in the slice header. Furthermore, the dimensions / configuration of subpicture 510 can be coded in the SPS along with the subpicture ID. Thus, the decoder can retrieve the configuration of subpicture 510 from the SPS based on the subpicture ID and position slice 515 in subpicture 510 without referring to the complete picture 500. Consequently, rewriting the slice header can be omitted when subpicture 510 is extracted, which significantly reduces resource usage in the encoder, decoder, and / or corresponding slicer.

[0090] Once picture 500 is divided into CTB518 and / or CTU519, CTB518 and / or CTU519 can be further divided into coding blocks. The coding blocks can then be coded according to intra-prediction and / or inter-prediction. This disclosure also includes improvements related to the inter-prediction mechanism. Inter-prediction can be performed in several different modes that can operate according to unidirectional inter-prediction and / or bidirectional inter-prediction.

[0091] Figure 6 is a schematic diagram showing an example of a one-way interpretation 600 performed to determine motion vectors (MV) in, for example, the block compression step 105, the block decoding step 113, the motion estimation component 221, the motion compensation component 219, the motion compensation component 321 and / or the motion compensation component 421. For example, the one-way interpretation 600 can be used to determine motion vectors for encoded and / or decoded blocks created when segmenting a picture such as picture 500.

[0092] The one-way interpretation 600 uses a reference frame 630 containing a reference block 631 to predict the current block 611 within the current frame 610. The reference frame 630 may be positioned in time after the current frame 610 (e.g., as a subsequent reference frame), as shown in the diagram, but in some examples it may be positioned in time before the current frame 610 (e.g., as a preceding reference frame). The current frame 610 is an exemplary frame / picture to be encoded / decoded at a particular time. The current frame 610 contains objects in the current block 611 that match objects in the reference block 631 of the reference frame 630. The reference frame 630 is the frame used as a reference for encoding the current frame 610, and the reference block 631 is the block in the reference frame 630 that contains objects that are also included in the current block 611 of the current frame 610.

[0093] The current block 611 is a coding unit that is being coded / decoded at a specified point in the coding process. The current block 611 may be an entire segmented block or a subblock when using the affine interpretation mode. The current frame 610 is separated from the reference frame 630 by some temporal distance (TD) 633. TD 633 represents the amount of time between the current frame 610 and the reference frame 630 in the video sequence and may be measured in units of frames. Predictive information about the current block 611 may refer to the reference frame 630 and / or the reference block 631 by a reference index that indicates the direction and temporal distance between frames. Over the period represented by TD 633, objects in the current block 611 move from their position in the current frame 610 to another position in the reference frame 630 (e.g., the position of the reference block 631). For example, an object may move along a motion trajectory 613, which is the direction of the object's movement over time. The motion vector 635 describes the direction and magnitude of the object's motion along the motion trajectory 613 across TD633. Thus, the encoded motion vector 635, the reference block 631, and the residual, which includes the difference between the current block 611 and the reference block 631, provide sufficient information to reconstruct the current block 611 and position it within the current frame 610.

[0094] Figure 7 is a schematic diagram showing an example of a bidirectional interpretation 700 performed to determine the motion vector (MV) in, for example, the block compression step 105, the block decoding step 113, the motion estimation component 221, the motion compensation component 219, the motion compensation component 321, and / or the motion compensation component 421. For example, the bidirectional interpretation 700 can be used to determine the motion vector for the encoded and / or decoded blocks created when partitioning a picture such as picture 500.

[0095] The bidirectional interpretation 700 is similar to the unidirectional interpretation 600, but uses a pair of reference frames to predict the current block 711 within the current frame 710. Thus, the current frame 710 and the current block 711 are substantially the same as the current frame 610 and the current block 611, respectively. The current frame 710 is temporally positioned between a preceding reference frame 720 that occurs before the current frame 710 in the video sequence and a succeeding reference frame 730 that occurs after the current frame 710 in the video sequence. The preceding reference frame 720 and the succeeding reference frame 730 are otherwise substantially the same as the reference frame 630.

[0096] The current block 711 is matched with the preceding reference block 721 in the preceding reference frame 720 and the subsequent reference block 731 in the subsequent reference frame 730. Such matching indicates that, in the course of the video sequence, an object moves from the position of the preceding reference block 721 along a motion trajectory 713 through the current block 711 to the position of the subsequent reference block 731. The current frame 710 is separated from the preceding reference frame 720 by some preceding time distance (TD0) 723 and from the subsequent reference frame 730 by some subsequent time distance (TD1) 733. TD0 723 is the amount of time in frames between the preceding reference frame 720 and the current frame 710 in the video sequence. TD1 733 is the amount of time in frames between the current frame 710 and the subsequent reference frame 730 in the video sequence. Therefore, the object moves along the motion path 713 from the preceding reference block 721 to the current block 711 over the period indicated by TD0 723. The object also moves along the motion path 713 from the current block 711 to the subsequent reference block 731 over the period indicated by TD1 733. Predictive information about the current block 711 may refer to the preceding reference frame 720 and / or the preceding reference block 721 and the subsequent reference frame 730 and / or the subsequent reference block 731 by a pair of reference indices indicating the direction and time distance between frames.

[0097] The preceding motion vector (MV0) 725 describes the direction and magnitude of the object's motion along the motion trajectory 713 across TD0 723 (for example, between the preceding reference frame 720 and the current frame 710). The succeeding motion vector (MV1) 735 describes the direction and magnitude of the object's motion along the motion trajectory 713 across TD1 733 (for example, between the current frame 710 and the succeeding reference frame 730). Therefore, in the bidirectional interpretation 700, the current block 711 can be coded and reproduced using the preceding reference block 721 and / or the succeeding reference block 731, MV0 725 and MV1 735.

[0098] In both merge mode and advanced motion vector prediction (AMVP) mode, the candidate list is generated by adding candidate motion vectors to the candidate list in the order defined by the candidate list determination pattern. Such candidate motion vectors may include motion vectors from one-way interpretation 600, two-way interpretation 700, or a combination thereof. Specifically, motion vectors are generated for adjacent blocks when such blocks are encoded. Such motion vectors are added to the candidate list for the current block, and motion vectors for the current block are selected from the candidate list. The motion vectors can then be signaled as indices of the selected motion vectors in the candidate list. The decoder can construct the candidate list using the same process as the encoder and can determine the motion vectors selected from the candidate list based on the signaled indices. Thus, the candidate motion vectors include motion vectors generated according to one-way interpretation 600 and / or two-way interpretation 700, depending on which technique is used when such adjacent blocks are encoded.

[0099] Figure 8 is a schematic diagram showing an example 800 in which the current block 801 is coded based on candidate motion vectors from an adjacent coded block 802. The operation method 100 of the encoder 300 and / or decoder 400, and / or the use of the functions of the codec system 200, allows the adjacent block 802 to be used to generate a candidate list. Such a candidate list can be used in interpretation by one-way interpretation 600 and / or bidirectional interpretation 700. The candidate list can then be used to encode / decode the current block 801, which may be generated by segmenting a picture such as picture 500.

[0100] Current block 801 is, depending on the example, a block that is encoded by the encoder or decoded by the decoder at a given time. Coated block 802 is a block that has already been encoded at a given time. Therefore, coded block 802 is potentially available for use when generating the candidate list. Current block 801 and coded block 802 may be contained in a common frame and / or in a temporally adjacent frame. When coded block 802 is contained in a common frame with current block 801, coded block 802 contains a boundary that is immediately adjacent (e.g., touching) to the boundary of current block 801. When coded block 802 is contained in a temporally adjacent frame, coded block 802 is located in the temporally adjacent frame at the same position as current block 801 in the current frame. The candidate list can be generated by adding motion vectors from coded block 802 as candidate motion vectors. Current block 801 can then be coded by selecting a candidate motion vector from the candidate list and signaling the index of the selected candidate motion vector.

[0101] Figure 9 is a schematic diagram showing an exemplary pattern 900 for determining a candidate list of motion vectors. Specifically, the operation method 100 of the encoder 300 and / or decoder 400, and / or the use of the functions of the codec system 200, can be used to use the candidate list determination pattern 900 when generating a candidate list 911 for encoding the current block 801 separated from picture 500. The resulting candidate list 911 may be a merged candidate list or an AMVP candidate list, which can be used in interpretation by one-way interpretation 600 and / or bidirectional interpretation 700.

[0102] When encoding the current block 901, the candidate list determination pattern 900 searches for valid candidate motion vectors at positions 905, indicated as A0, A1, B0, B1, and / or B2, within the same picture / frame as the current block. The candidate list determination pattern 900 may also search for valid candidate motion vectors at the same positions in block 909. The same-positioned block 909 is block 901 at the same position as the current block 901, but in a temporally adjacent picture / frame. The candidate motion vectors can then be placed in the candidate list 911 in a predetermined check order. Thus, the candidate list 911 is a procedurally generated list of indexed candidate motion vectors.

[0103] Candidate list 911 can be used to select motion vectors for performing interpretation on the current block 901. For example, the encoder can obtain a sample of a reference block pointed to by a candidate motion vector from candidate list 911. The encoder can then select a candidate motion vector that points to the reference block that best matches the current block 901. The index of the selected candidate motion vector can then be encoded to represent the current block 901. In some cases, a candidate motion vector points to a reference block containing a partial reference sample 915. In this case, an interpolation filter 913 can be used to reconstruct the complete reference sample 915 to support motion vector selection. The interpolation filter 913 is a filter capable of upsampling a signal. Specifically, the interpolation filter 913 is a filter that accepts a partial / lower quality signal as input and is capable of determining an approximation of a more complete / higher quality signal. Thus, the interpolation filter 913 can be used, in certain cases, to obtain a complete set of reference samples 915 for use when selecting a reference block for the current block 901, and therefore when selecting a motion vector to encode the current block 901.

[0104] The above mechanism for coding blocks based on interpretation using a candidate list can cause certain errors when subpictures such as subpicture 510 are used. Specifically, problems can arise when the current block 901 is contained in the current subpicture, but the motion vector points to a reference block that is at least partially located in an adjacent subpicture. In such cases, the current subpicture may be extracted to be presented without the adjacent subpicture. When this occurs, the portion of the reference block in the adjacent subpicture may not be transmitted to the decoder, and therefore the reference block may not be available to decode the current block 901. When this occurs, the decoder does not have access to enough data to decode the current block 901.

[0105] This disclosure provides a mechanism to address this problem. In one example, a flag is used to indicate that the current subpicture should be treated as a picture. This flag can be set to support separate extraction of subpictures. Specifically, when the flag is set, the current subpicture should be encoded without referencing data in other subpictures. In this case, the current subpicture is treated as a picture in the sense that it is coded separately from other subpictures and can be displayed as a separate picture. Thus, this flag may be denoted as subpic_treated_as_pic_flag[i], where i is the index of the current subpicture. When the flag is set, motion vector candidates (also known as motion vector predictors) obtained from block 909 at the same location include only motion vectors pointing within the current subpicture. Any motion vector predictors pointing outside the current subpicture are excluded from the candidate list 911. This ensures that motion vectors pointing outside the current subpicture are not selected and that associated errors are avoided. This example applies in particular to motion vectors from block 909 at the same location. Motion vectors from search position 905 within the same picture / frame may be corrected by separate mechanisms, as described below.

[0106] Another example may be used to address the search position 905 when the current subpicture is treated as a picture (for example, when subpic_treated_as_pic_flag[i] is set). When the current subpicture is treated as a picture, it should be extracted without referencing other subpictures. An exemplary mechanism relates to an interpolation filter 913. The interpolation filter 913 can be applied to a sample at one location to interpolate (e.g., predict) related samples at other locations. In this example, the motion vector from the coded block at the search position 905 may point to such a reference sample 915, as long as the interpolation filter 913 can interpolate the reference sample 915 outside the current subpicture based only on the reference sample 915 from the current subpicture. Thus, this example uses a clipping function that is applied when applying the interpolation filter 913 to a candidate motion vector from the search position 905 from the same picture. This clipping function clips data from adjacent subpictures and therefore removes such data as input to the interpolation filter 913 when determining the reference sample 915 pointed to by the motion vector candidate. This technique maintains separation between subpictures during encoding so as to support separate extraction and decoding when subpictures are treated as pictures. The clipping function may be applied to a lumane sample bilinear interpolation process, a lumane sample 8-tap interpolation filtering process, and / or a chromane sample interpolation process.

[0107] Figure 10 is a block diagram illustrating an exemplary in-loop filter 1000. The in-loop filter 1000 may be used to implement in-loop filters 225, 325 and / or 425. Furthermore, the in-loop filter 1000 may be applied in the encoder and decoder when performing method 100. In addition, the in-loop filter 1000 can be applied to filter the current block 801 separated from picture 500, which may be coded according to one-way interpretation 600 and / or two-way interpretation 700 based on a candidate list generated according to pattern 900. The in-loop filter 1000 includes a deblocking filter 1043, an SAO filter 1045 and an adaptive loop filter (ALF) 1047. The filters of the in-loop filter 1000 are applied sequentially to the reconstructed image block in the encoder (e.g., before being used as a reference block) and in the decoder before display.

[0108] The deblocking filter 1043 is configured to remove the edges of block shapes created by block-based inter and intra predictions. The deblocking filter 1043 scans an image portion (e.g., an image slice) to find discontinuities in chroma and / or luma values ​​occurring at the block boundaries. The deblocking filter 1043 then applies a smoothing function to the block boundaries to remove such discontinuities. The intensity of the deblocking filter 1043 may be modified depending on the spatial activity (e.g., variance of luma / chroma components) occurring in the region adjacent to the block boundary.

[0109] The SAO filter 1045 is configured to remove artifacts related to sample distortion caused by the encoding process. In the encoder, the SAO filter 1045 categorizes the deblocked samples of the reconstructed image into several categories based on the relative deblocking edge shape and / or direction. An offset is then determined based on the category and added to the samples. The offset is then encoded into a bitstream and used by the SAO filter 1045 in the decoder. The SAO filter 1045 removes banding artifacts (bands of values ​​rather than smooth transitions) and ringing artifacts (spurious signals near sharp edges).

[0110] The ALF1047 is configured in the encoder to compare the reconstructed image with the original image. The ALF1047 determines coefficients that describe the difference between the reconstructed image and the original image, for example, via a Wiener-based adaptive filter. These coefficients are encoded into a bitstream and used by the ALF1047 in the decoder to remove the difference between the reconstructed image and the original image.

[0111] Image data filtered by the in-loop filter 1000 is output to picture buffer 1023, which is substantially the same as the decoded picture buffers 223, 323 and / or 423. As described above, the deblocking filter 1043, the SAO filter 1045 and / or ALF 1047 can be turned off at subpicture boundaries and / or tile boundaries by flags such as the loop_filter_across_subpic_enabled flag and / or loop_filter_across_tiles_enabled_flag, respectively.

[0112] Figure 11 is a schematic diagram showing an exemplary bitstream 1100 including coding tool parameters to support decoding the subpictures of a picture. For example, bitstream 1100 can be generated by codec system 200 and / or encoder 300 for decoding by codec system 200 and / or decoder 400. As another example, bitstream 1100 may be generated by encoder in step 109 of method 100 for use by decoder in step 111. Furthermore, bitstream 1100 may include an encoded picture 500, a corresponding subpicture 510, and / or associated coded blocks such as current blocks 801 and / or 901, which may be coded according to one-way interpretation 600 and / or two-way interpretation 700 based on a candidate list generated according to pattern 900. Bitstream 1100 may also include parameters for configuring an in-loop filter 1000.

[0113] The bitstream 1100 includes a sequence parameter set (SPS) 1110, multiple picture parameter sets (PPS) 1111, multiple slice headers 1115, and image data 1120. The SPS 1110 contains sequence data common to all pictures in the video sequence contained in the bitstream 1100. Such data may include picture size, bit depth, coding tool parameters, bitrate limits, etc. The PPS 1111 contains parameters that apply to the entire picture. Therefore, each picture in the video sequence may refer to the PPS 1111. While each picture refers to the PPS 1111, it should be noted that in some examples, a single PPS 1111 may contain data for multiple pictures. For example, multiple similar pictures may be coded according to similar parameters. In such cases, a single PPS 1111 may contain data for such similar pictures. The PPS 1111 can indicate the coding tools, quantization parameters, offsets, etc., available for slices in the corresponding picture. The slice header 1115 contains parameters specific to each slice within the picture. Therefore, a video sequence may have one slice header 1115 per slice. The slice header 1115 may include slice type information, picture order count (POC), reference picture list, prediction weights, tile entry point, deblocking parameters, etc. It should be noted that the slice header 1115 may also be referred to as a tile group header in some contexts.

[0114] Image data 1120 includes video data encoded according to inter-prediction and / or intra-prediction, and corresponding transformed and quantized residual data. For example, a video sequence includes multiple pictures coded as image data. A picture is a single frame of a video sequence and is therefore generally displayed as a single unit when displaying a video sequence. However, subpictures may be displayed to implement certain technologies such as virtual reality, picture-in-picture, etc. Each picture refers to a PPS 1111. A picture is divided into subpictures, tiles, and / or slices, as described above. In some systems, a slice is called a tile group containing tiles. A slice and / or tile group of tiles refers to a slice header 1115. A slice is further divided into CTUs and / or CTBs. A CTU / CTB is further divided into coding blocks based on a coding tree. The coding blocks can then be coded / decoded according to a prediction mechanism.

[0115] The parameter set within bitstream 1100 contains various data that can be used to implement the examples described herein. To support the implementation of the first example, SPS1110 of bitstream 1100 contains a subpicture flag 1131 for a given subpicture that is treated as a picture. In some examples, the subpicture flag 1131 is denoted as subpic_treated_as_pic_flag[i], where i is the index of the subpicture associated with the flag. For example, the subpicture flag 1131 may be set to equal to 1 to specify that the i-th subpicture of each coded picture in the coded video sequence (in image data 1120) is treated as a picture in the decoding process, excluding the in-loop filtering operation. The subpicture flag 1131 may be used when the current subpicture in the current picture is coded according to interpretation. When the flag 1131 of a subpicture treated as a picture is set to indicate that the current subpicture is treated as a picture, the candidate list of candidate motion vectors for the current block can be determined by excluding motion vectors of the same position that are included in the block at the same position and point outside the current subpicture from the candidate list. This ensures that when the current subpicture is extracted separately from other subpictures, motion vectors pointing outside the current subpicture are not selected, and associated errors are avoided.

[0116] In some examples, the list of candidate motion vectors for the current block is determined according to time-lumen motion vector prediction. For example, time-lumen motion vector prediction may be used when the current block is a luma block of luma samples, the selected current motion vector for the current block is a time-lumen motion vector pointing to a reference luma sample in a reference block, and the current block is coded based on the reference luma sample. In such cases, time-lumen motion vector prediction is performed as follows: xColBr=xCb+cbWidth; yColBr = yCb + cbHeight; rightBoundaryPos=subpic_treated_as_pic_flag[SubPicIdx]? SubPicRightBoundaryPos:pic_width_in_luma_samples-1; and botBoundaryPos=subpic_treated_as_pic_flag[SubPicIdx]? SubPicBotBoundaryPos:pic_height_in_luma_samples-1 Here, xColBr and yColBR specify the positions of blocks at the same location, xCb and yCb specify the top-left sample of the current block relative to the top-left sample of the current picture, cbWidth is the width of the current block, cbHeight is the height of the current block, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and pic_width_in_luma_samples is the width of the current picture measured in luma samples. pic_height_in_luma_samples is the height of the current picture measured in luma samples, botBoundaryPos is the calculated position of the lower boundary of the subpicture, rightBoundaryPos is the calculated position of the right boundary of the subpicture, SubPicIdx is the index of the subpicture, and when yCb>>CtbLog2SizeY is not equal to yColBr>>CtbLog2SizeY, the motion vectors at the same position are excluded, and CtbLog2SizeY indicates the size of the coding tree block.

[0117] The flag 1131 for a subpicture treated as a picture may also be used for the implementation of the second example. As in the first example, the flag 1131 for a subpicture treated as a picture may be used when the current subpicture within the current picture is coded according to interpretation. In this example, a motion vector can be determined for the current block of the subpicture (e.g., from a candidate list). When the flag 1131 for a subpicture treated as a picture is set, a clipping function can be applied to a sample position within the reference block. A sample position is a position within the picture that may contain a single sample containing a luma value and / or a pair of chroma values. An interpolation filter can then be applied when the motion vector points outside the current subpicture. This clipping function ensures that the interpolation filter does not depend on data from adjacent subpictures in order to maintain separation between subpictures to support separate extractions.

[0118] The clipping function can be applied in the luma sample bilinear interpolation process. The luma sample bilinear interpolation process may receive an input that includes the luma position in all sample units (xIntL, yIntL). The luma sample bilinear interpolation process outputs a predicted luma sample value (predSampleLXL). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following applies: xInti = Clip3(SubPicLeftBoundaryPos, SubPicRightBoundaryPos, xIntL + i), and yInti=Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yIntL+i) Here, subpic_treated_as_pic_flag is a flag set to indicate that the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and Clip3 is a clipping function that follows the following.

number

[0119] The clipping function can also be applied in the Luma Sample 8-Tap Interpolation Filtering process. The Luma Sample 8-Tap Interpolation Filtering process receives an input containing the Luma position in all sample units (xIntL, yIntL). The Luma Sample Bilinear Interpolation process outputs the predicted Luma Sample value (predSampleLXL). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following holds: xInti=Clip3(SubPicLeftBoundaryPos,SubPicRightBoundaryPos,xIntL+i-3), and yInti=Clip3(SubPicTopBoundaryPos,SubPicBotBoundaryPos,yIntL+i-3) Here, subpic_treated_as_pic_flag is a flag set to indicate that the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, and Clip3 is as described above.

[0120] The clipping function can also be applied in the chroma sample interpolation process. The chroma sample interpolation process receives an input containing the chroma position in all sample units (xIntC, yIntC). The chroma sample interpolation process outputs a predicted chroma sample value (predSampleLXC). The clipping function is applied to the sample position as follows: When subpic_treated_as_pic_flag[SubPicIdx] is equal to 1, the following holds: xInti = Clip3(SubPicLeftBoundaryPos / SubWidthC,SubPicRightBoundaryPos / SubWidthC,xIntC+i), and yInti=Clip3(SubPicTopBoundaryPos / SubHeightC,SubPicBotBoundaryPos / SubHeightC,yIntC+i) Here, subpic_treated_as_pic_flag is a flag set to indicate that the subpicture is treated as a picture, SubPicIdx is the index of the subpicture, xInti and yInti are the clipped sample positions at index i, SubPicRightBoundaryPos is the position of the right boundary of the subpicture, SubPicLeftBoundaryPos is the position of the left boundary of the subpicture, SubPicTopBoundaryPos is the position of the top boundary of the subpicture, SubPicBotBoundaryPos is the position of the bottom boundary of the subpicture, SubWidthC and SubHeightC indicate the horizontal and vertical sampling rate ratios between the luminous and chroma samples, and Clip3 is as described above.

[0121] The subpicture loop filter enable flag 1132 within SPS1110 may be used for the implementation of the third example. The subpicture loop filter enable flag 1132 may be set to control whether filtering is used across the boundaries of a specified subpicture. For example, the subpicture loop filter enable flag 1132 may be indicated as loop_filter_across_subpic_enabled_flag. The subpicture loop filter enable flag 1132 may be set to 1 to specify that the in-loop filtering operation can be performed across the subpicture boundaries, or to 0 to specify that the in-loop filtering operation will not be performed across the subpicture boundaries. Thus, the filtering operation may or may not be performed across subpicture boundaries based on the value of the subpicture loop filter enable flag 1132. The filtering operation may include applying the deblocking filter 1043, ALF 1047, and / or SAO filter 1045. Thus, the filters can be turned off for subpictures encoded for separate extraction, or turned on for subpictures encoded for display as a group.

[0122] The flag 1134 for enabling loop filtering across tiles in PPS1111 may be used for the implementation of the fourth example. The flag 1134 for enabling loop filtering across tiles may be set to control whether filtering is used across the boundaries of a specified tile. For example, the flag 1134 for enabling loop filtering across tiles may be indicated as loop_filter_across_tiles_enabled_flag. The flag 1134 for enabling loop filtering across tiles may be set to 1 to specify that the in-loop filtering operation can be performed across tile boundaries, or to 0 to specify that the in-loop filtering operation will not be performed across tile boundaries. Thus, the filtering operation may or may not be performed across a specified tile boundary based on the value of the flag 1134 for enabling loop filtering across tiles. The filtering operation may include applying the deblocking filter 1043, ALF 1047, and / or SAO filter 1045.

[0123] The subpicture data 1133 in SPS1110 may be used for the implementation of the fifth example. The subpicture data 1133 may include the width, height, and offset for each subpicture in the image data 1120. For example, the width and height of each subpicture can be described in the subpicture data 1133 in units of CTB. In some examples, the width and height of the subpicture are stored in the subpicture data 1133 as subpic_width_minus1 and subpic_height_minus1, respectively. Furthermore, the offset of each subpicture can be described in the subpicture data 1133 in units of CTU. For example, the offset of each subpicture can be specified as the vertical and horizontal positions of the top-left CTU of the subpicture. Specifically, the offset of a subpicture can be specified as the difference between the top-left CTU of the picture and the top-left CTU of the subpicture. In some examples, the vertical and horizontal positions of the top-left CTU of a subpicture are stored in the subpicture data 1133 as subpic_ctu_top_left_y and subpic_ctu_top_left_x, respectively. The implementation in this example describes the subpicture in the subpicture data 1133 in terms of the CTB / CTU rather than the tile. This allows the subpicture to be used even when the tile is not used in the corresponding picture / subpicture.

[0124] The subpicture data 1133 in SPS1110, the slice address 1136 in the slice header 1115, and the slice subpicture ID 1135 in the slice header 1115 may be used for the implementation of the sixth example. The subpicture data 1133 may be implemented as described in the implementation of the fifth example. The slice address 1136 may contain a subpicture-level slice index of the slice associated with the slice header 1115 (e.g., image data 1120). For example, the slice is indexed based on the position of the slice in the subpicture rather than based on the position of the slice in the picture. The slice address 1136 may be stored in the slice_address variable. The slice subpicture ID 1135 contains the ID of the subpicture containing the slice associated with the slice header 1115. Specifically, the slice subpicture ID 1135 may refer to the description of the corresponding subpicture in the subpicture data 1133 (e.g., width, height, and offset). The slice subpicture ID 1135 may be stored in the `slice_subpic_id` variable. Thus, the slice address 1136 is signaled as an index based on the position of the slice within the subpicture indicated by the subpicture ID 1135, as described in the subpicture data 1133. In this way, the position of the slice within a subpicture can be determined even when subpictures are extracted separately and other subpictures are omitted from the bitstream 1100. This is because this addressing scheme isolates the address of each subpicture from that of other subpictures. Therefore, the slice header 1115 does not need to be rewritten when the subpicture is extracted, as is required in addressing schemes where slices are addressed based on the position of the slice within a picture. It should be noted that this technique may be used when the slice is a rectangular slice (as opposed to a raster scan slice). For example, the `rect_slice_flag` in PPS 1111 can be set to equal to 1 to indicate that the slice is a rectangular slice.

[0125] The following are exemplary implementations of subpictures used in some video coding systems. Information related to subpictures that may exist in the CVS may be signaled in the SPS. Such signaling may include the following information: The number of subpictures present in each picture of the CVS may be included in the SPS. In the context of the SPS or CVS, subpictures at the same location for all access units (AUs) may collectively be called a subpicture sequence. Loops for specifying further information related to the characteristics of each subpicture may also be included in the SPS. Such information may include the subpicture identifier, the subpicture location (e.g., the offset distance between the upper-left corner luma sample of the subpicture and the upper-left corner luma sample of the picture), and the subpicture size. Furthermore, the SPS may also be used to signal whether each subpicture is a motion-constrained subpicture, which is a subpicture containing an MCTS. Profile, hierarchy, and level information for each subpicture may be included in the bitstream unless such information can be derived separately. Such information may be used for profile, hierarchy, and level information for the extracted bitstream created by extracting subpictures from the original bitstream containing the entire picture. The profile and hierarchy of each subpicture may be derived to be the same as the original profile and hierarchy. The level for each subpicture may be explicitly signaled. Such signaling may be present in the loop described above. Sequence-level hypothetical reference decoder (HRD) parameters may be signaled in the video usability information (VUI) portion of the SPS for each subpicture (or equivalently, each subpicture sequence).

[0126] When a picture is not divided into two or more subpictures, the characteristics of the subpictures (e.g., position, size, etc.) do not need to be signaled in the bitstream, except for the subpicture ID. When subpictures within a picture in a CVS are extracted, each access unit in the new bitstream does not need to contain a subpicture. This is because the resulting image data in each AU in the new bitstream is not divided into multiple subpictures. Therefore, subpicture characteristics such as position and size may be omitted from the SPS because such information can be derived from the picture characteristics. However, subpicture identification is still signaled because this ID may be referenced by the video coding layer (VCL) NAL units / tile groups contained in the extracted subpictures. To reduce resource usage, changing the subpicture ID should be avoided when extracting subpictures.

[0127] The position (x-offset and y-offset) of a subpicture within a picture can be signaled in units of luma samples, which may represent the distance between the upper-left luma sample of the subpicture and the upper-left luma sample of the picture. In another example, the position of a subpicture within a picture can be signaled in units of the minimum coding luma block size (MinCbSizeY), which may represent the distance between the upper-left luma sample of the subpicture and the upper-left luma sample of the picture. In yet another example, the units of the subpicture position offset may be explicitly indicated by a syntax element in the parameter set, which may be CtbSizeY, MinCbSizeY, luma samples, or other values. The codec may require that the width of a subpicture be an integer multiple of the luma CTU size (CtbSizeY) when the right boundary of the subpicture does not coincide with the right boundary of the picture. Similarly, the codec may further require that the height of a subpicture be an integer multiple of CtbSizeY when the bottom boundary of the subpicture does not coincide with the bottom boundary of the picture. The codec may also require that the subpicture be positioned at the rightmost position of the picture when the width of the subpicture is not an integer multiple of the Luma CTU size. Similarly, the codec may require that the subpicture be positioned at the bottom of the picture when the height of the subpicture is not an integer multiple of the Luma CTU size. When the width of the subpicture is signaled in units of the Luma CTU size, and the width of the subpicture is not an integer multiple of the Luma CTU size, the actual width in the Luma sample may be derived based on the offset position of the subpicture, the width of the subpicture in Luma CTU size, and the width of the picture in the Luma sample. Similarly, when the height of a subpicture is signaled in units of Luma CTU size, and the height of the subpicture is not an integer multiple of Luma CTU size, the actual height in the Luma sample can be derived based on the offset position of the subpicture, the height of the subpicture in Luma CTU size, and the height of the picture in the Luma sample.

[0128] For any subpicture, the subpicture ID may differ from the subpicture index. The subpicture index may be the index of the subpicture that is signaled in the subpicture loop within the SPS. Alternatively, the subpicture index may be the index assigned to the picture in the order of the subpicture raster scan. When the value of the subpicture ID for each subpicture is the same as its subpicture index, the subpicture ID may be signaled or derived. When the subpicture ID for each subpicture differs from its subpicture index, the subpicture ID is explicitly signaled. The number of bits for signaling the subpicture ID may be signaled (e.g., in the SPS) with the same set of parameters that include the subpicture characteristics. Some values ​​for the subpicture ID may be reserved for specific purposes. Such reservations of values ​​may be as follows: When a tile group / slice header includes a subpicture ID to specify which subpictures contain a tile group, the value 0 may be reserved and not used for subpictures to ensure that the first few bits of the tile group / slice header are not all zero and to avoid generating emulation prevention code. When the subpictures of a picture do not cover the entire area of ​​the picture without overlap and gaps, the value (e.g., value 1) may be reserved for tile groups that are not part of any subpicture. Alternatively, the subpicture IDs for the remaining areas may be explicitly signaled. The number of bits for signaling subpicture IDs may be constrained as follows: The range of values ​​should include the reserved value for subpicture IDs and be sufficient to uniquely identify all subpictures within a picture. For example, the minimum number of bits for a subpicture ID can be the value of Ceil(Log2(number of subpictures in the picture + number of reserved subpicture IDs)).

[0129] The group of subpictures within a loop may be required to cover the entire picture without gaps and overlaps. When this constraint is applied, a flag exists for each subpicture to specify whether it is a motion-constrained subpicture, meaning that the subpicture can be extracted. Alternatively, the group of subpictures does not have to cover the entire picture, but there does not have to be any overlap between the subpictures of the picture.

[0130] The subpicture ID may reside immediately after the NAL unit header to assist the subpicture extraction process, so that the extractor does not need to understand the rest of the NAL unit bits. For VCL NAL units, the subpicture ID may reside in the first bit of the tile group header. For non-VCL NAL units, the following conditions may apply: The subpicture ID does not need to reside immediately after the NAL unit header for SPS. With respect to PPS, the subpicture ID does not need to reside immediately after the NAL unit header when all tile groups of the same picture are constrained to refer to the same PPS. On the other hand, if tile groups of the same picture are allowed to refer to different PPS, the subpicture ID may reside in the first bit of the PPS (e.g., immediately after the PPS NAL unit header). In this case, two different tile groups of the same picture are not allowed to share the same PPS. Alternatively, when tile groups of the same picture are allowed to refer to different PPS, and different tile groups of the same picture are allowed to share the same PPS, the subpicture ID does not exist in the PPS syntax. Alternatively, when tile groups of the same picture are allowed to reference different PPSs, and when different tile groups of the same picture are allowed to share the same PPS, a list of subpicture IDs exists in the PPS syntax. This list indicates the subpictures to which the PPS applies. For other non-VCL NAL units, if the non-VCL unit applies at or above the picture level (e.g., access unit delimiter, end of sequence, end of bitstream, etc.), the subpicture ID does not need to be immediately after its NAL unit header. Otherwise, the subpicture ID may be immediately after the NAL unit header.

[0131] Tile divisions within individual subpictures may be signaled by PPS, but tile groups within the same picture are allowed to reference different PPS. In this case, tiles are grouped within each subpicture rather than across the picture. Therefore, the concept of tile grouping in such cases involves dividing subpictures into tiles. Alternatively, a Sub-Picture Parameter Set (SPPS) may be used to describe tile divisions within individual subpictures. The SPPS references the SPS by using syntax elements that reference the SPS ID. The SPPS may also include subpicture IDs. For the purpose of subpicture extraction, the syntax element that references the subpicture ID is the first syntax element in the SPPS. The SPPS includes a tile structure indicating the number of columns, the number of rows, uniform tile spacing, etc. The SPPS may also include flags to indicate whether a loop filter is valid across the relevant subpicture boundaries. Alternatively, subpicture characteristics for each subpicture may be signaled by the SPPS instead of the SPS. Tile divisions within individual subpictures may be signaled by PPS, but tile groups within the same picture are permitted to reference different PPS. Once activated, SPPS may persist throughout a sequence of consecutive AUs in decoding order, but may be deactivated / activated in AUs that are not the start of CVS. Multiple SPPS may be active at any point during the decoding process of a single-layer bitstream having multiple subpictures, and SPPS may be shared by different subpictures of an AU. Alternatively, SPPS and PPS can be merged into a single parameter set. For this to occur, all tile groups contained within the same subpicture may be constrained to reference the same parameter set resulting from the merge between SPPS and PPS.

[0132] The number of bits used to signal the subpicture ID may be signaled in the NAL unit header. Such information, when present, assists the subpicture extraction process in parsing the subpicture ID value for the beginning of the NAL unit payload (e.g., the first few bits immediately following the NAL unit header). For such signaling, some of the reserved bits in the NAL unit header may be used to avoid increasing the length of the NAL unit header. The number of bits for such signaling should cover the value of sub-picture-ID-bit-len. For example, four of the seven reserved bits in the NAL unit header of a VVC may be used for this purpose.

[0133] When decoding a subpicture, the position of each coding tree block, indicated as the vertical CTB position (xCtb) and horizontal CTB position (yCtb), is adjusted to the actual luma sample position in the picture, rather than the luma sample position within the subpicture. Thus, everything is decoded as if it were located in the picture rather than the subpicture, thus avoiding the extraction of identical subpictures from each reference picture. To adjust the coding tree block positions, the variables SubpictureXOffset and SubpictureYOffset are derived based on the subpicture positions (subpic_x_offset and subpic_y_offset). The values ​​of these variables are added to the x and y coordinate values ​​of the luma sample positions of each coding tree block within the subpicture, respectively. The subpicture extraction process can be defined as follows: The input to the process contains the target subpicture to be extracted. This can be in the form of a subpicture ID or a subpicture position. When the input is a subpicture position, the associated subpicture ID can be determined by parsing the subpicture information in the SPS. The following applies to non-VCL NAL units: Syntax elements within the SPS related to picture size and level are updated with the subpicture size and level information. The following non-VCL NAL units, namely PPS, access unit delimiter (AUD), end of sequence (EOS), end of bitstream (EOB), and any other non-VCL NAL units applicable to picture level or higher, are not modified by extraction. Any remaining non-VCL NAL units whose subpicture ID is not equal to the target subpicture ID are removed. VCL NAL units whose subpicture ID is not equal to the target subpicture ID are also removed.

[0134] Subpicture nesting SEI messages may be used to nest AU-level or subpicture-level SEI messages for a set of subpictures. The data carried in a subpicture nesting SEI message may include buffering periods, picture timings, and non-HRD SEI messages. The syntax and semantics of this SEI message may be as follows. For system operation such as in an omnidirectional media format (OMAF) environment, a set of subpicture sequences covering a viewport may be requested and decoded by an OMAF player. Therefore, a sequence-level SEI message may carry information about a set of subpicture sequences that also include rectangular or square picture areas. This information is available to the system and indicates the minimum decoding capability and the bitrate of the set of subpicture sequences. This information includes the bitstream level containing only the set of subpicture sequences, the bitrate of the bitstream, and optionally a specified subbitstream extraction process for the set of subpicture sequences.

[0135] The above implementation method has several problems. Signaling the width and height of the picture, and / or the width / height / offset of the subpictures is inefficient. More bits could be saved by signaling such information. When subpicture size and position information is signaled in the SPS, the PPS includes a tiled structure. Furthermore, the PPS can be shared by multiple subpictures of the same picture. Therefore, the range of values ​​for num_tile_columns_minus1 and num_tile_rows_minus1 should be specified more clearly. Furthermore, the semantics of the flag indicating whether a subpicture is motion-constrained or not are not clearly specified. Levels are required to be signaled for each subpicture sequence. However, signaling the level of a subpicture is not useful when subpicture sequences cannot be decoded independently. Furthermore, in some applications, several subpicture sequences should be decoded and rendered together with at least one other subpicture sequence. Therefore, signaling the level for a single such subpicture sequence may not be useful. Furthermore, determining the level value for each sub-picture can put a strain on the encoder.

[0136] The introduction of independently decodeable sub-picture sequences means that scenarios requiring independent extraction and decoding of specific regions of a picture may not work based on tile groups. Therefore, explicit signaling of tile group IDs may not be useful. Furthermore, the values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag, respectively, should be the same for all PPS referenced by the tile group header of the coded picture. This is because the active SPS should not change within the CVS, and the value of loop_filter_across_tiles_enabled_flag should be the same for all tiles in the picture for tile-based parallel processing. Whether or not to allow a mix of rectangular and raster scan tile groups within a picture should be explicitly specified. It should also be specified whether or not to allow sub-pictures that are part of different pictures and use the same sub-picture ID within the CVS in order to use different tile group modes. The derivation process for temporal motion vector prediction may not allow for the treatment of sub-picture boundaries as picture boundaries in temporal motion vector prediction (TMVP). Furthermore, the luma-sample bilinear interpolation process, luma-sample 8-tap interpolation filtering process, and chroma-sample interpolation process may not be configured to treat sub-picture boundaries as picture boundaries in motion compensation. Mechanisms for controlling deblocking, SAO, and ALF filtering operations at sub-picture boundaries should also be specified.

[0137] The introduction of independently decodeable sub-picture sequences means that `loop_filter_across_tile_groups_enabled_flag` may not be very useful. This is because turning off in-loop filtering for the purpose of parallel processing can sometimes be satisfied by setting `loop_filter_across_tile_groups_enabled_flag` to 0. Furthermore, turning off in-loop filtering to allow independent extraction and decoding of specific regions of a picture can also be satisfied by setting `loop_filter_across_sub_pic_enabled_flag` to 0. Therefore, further specifying a process to turn off in-loop filtering across tile group boundaries based on `loop_filter_across_tile_groups_enabled_flag` unnecessarily burdens the decoder and wastes bits. Moreover, the above decoding process may not allow turning off ALF filtering across tile boundaries.

[0138] Therefore, this disclosure includes a design for supporting subpicture-based video coding. A subpicture is a rectangular or square area within a picture that may or may not be decoded independently using the same decoding process as the picture. The description of the technique is based on the Versatile Video Coding (VVC) standard. However, the technique may also be applied to other video codec specifications.

[0139] In some examples, size units are signaled for the syntactic elements for picture width and height, and for the list of syntactic elements for subpicture width / height / offset_x / offset_y. All syntactic elements are signaled in the format xxx_minus1. For example, when the size unit is 64 luma samples, a width value of 99 specifies a picture width of 6400 luma samples. The same example applies to others of these syntactic elements. In other examples, one or more of the following may apply: Size units may be signaled for the syntactic elements for picture width and height in the format xxx_minus1. Such size units signaled for the list of syntactic elements for subpicture width / height / offset_x / offset_y may also be in the format xxx_minus1. In other examples, one or more of the following may apply. The size units for the list of syntax elements for picture width and sub-picture width / offset_x may be signaled in the format xxx_minus1. The size units for the list of syntax elements for picture height and sub-picture height / offset_y may be signaled in the format xxx_minus1. In other examples, one or more of the following may apply: The syntax elements for picture width and height in the format xxx_minus1 may be signaled in units of the minimum coding unit. The syntax elements for sub-picture width / height / offset_x / offset_y in the format xxx_minus1 may be signaled in units of CTU or CTB. The sub-picture width for each sub-picture at the right picture boundary may be derived. The sub-picture height for each sub-picture at the bottom picture boundary may be derived. All other values ​​of sub-picture width / height / offset_x / offset_y may be signaled in bitstream. In other examples, modes for signaling the width and height of subpictures and their positions within the picture may be added for cases where subpictures have uniform sizes.Subpictures have a uniform size when they contain the same subpicture row and subpicture column. In this mode, the number of subpicture rows, the number of subpicture columns, the width of each subpicture column, and the height of each subpicture row may all be signaled.

[0140] In other examples, signaling of subpicture width and height may not be included in the PPS. num_tile_columns_minus1 and num_tile_rows_minus1 should be in the range of 0 to 1024, or less than or equal to a single integer value. In other examples, when a subpicture that references the PPS has more than one tile, two syntax elements, conditional by the existence flag, may be signaled in the PPS. These syntax elements are used to signal subpicture width and height in CTB units and specify the size of all subpictures that reference the PPS.

[0141] In other examples, further information describing individual subpictures may also be signaled. A flag such as sub_pic_treated_as_pic_flag[i] may be signaled for each subpicture sequence to indicate whether the subpictures in the subpicture sequence are treated as pictures in the decoding process for purposes other than in-loop filtering. The level to which a subpicture sequence conforms may be signaled only when sub_pic_treated_as_pic_flag[i] is equal to 1. A subpicture sequence is a CVS of subpictures having the same subpicture ID. When sub_pic_treated_as_pic_flag[i] is equal to 1, the level of the subpicture sequence may also be signaled. This can be controlled by a flag for all subpicture sequences or by one flag for each subpicture sequence. In other examples, subbitstream extraction may be enabled without modifying the VCL NAL unit. This can be achieved by removing the signaling of explicit tile group IDs from the PPS. The semantics of `tile_group_address` are specified when `rect_tile_group_flag` is equal to what indicates a rectangular tile group. `tile_group_address` may also contain the tile group index of a tile group within a tile group in a subpicture.

[0142] In other examples, the values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag, respectively, are assumed to be the same for all PPS referenced by the tile group header of the coded picture. Other PPS syntax elements may differ for different PPS referenced by the tile group header of the coded picture. The value of single_tile_in_pic_flag may differ for different PPS referenced by the tile group header of the coded picture. Thus, some pictures in CVS may have only one tile, while some other pictures in CVS may have multiple tiles. This also allows some subpictures of a picture (e.g., very large ones) to have multiple tiles, while other subpictures of the same picture (e.g., very small ones) may have only one tile.

[0143] In other examples, a picture may contain a mixture of rectangular and raster scan tile groups. Therefore, some subpictures of a picture may use rectangular tile group mode, while others use raster scan tile group mode. This flexibility is beneficial in bitstream merging scenarios. Alternatively, the constraint may require that all subpictures of a picture use the same tile group mode. Subpictures from different pictures with the same subpicture ID in the CVS do not necessarily have to use different tile group modes. Subpictures from different pictures with the same subpicture ID in the CVS may use different tile group modes.

[0144] In another example, when sub_pic_treated_as_pic_flag[i] for a subpicture is equal to 1, the motion vectors at the same location for time motion vector prediction for the subpicture are restricted to originating from within the subpicture boundary. Thus, time motion vector prediction for a subpicture is treated as if the subpicture boundary were a picture boundary. Furthermore, to enable treating the subpicture boundary as a picture boundary in motion compensation for subpictures where sub_pic_treated_as_pic_flag[i] is equal to 1, the clipping operation is specified as part of the lumana sample bilinear interpolation process, the lumana sample 8-tap interpolation filtering process, and the chromana sample interpolation process.

[0145] In other examples, each subpicture is associated with a signaled flag such as loop_filter_across_sub_pic_enabled_flag. The flag is used to control the in-loop filtering behavior at the subpicture boundaries and the filtering behavior in the corresponding decoding process. The deblocking filter process does not have to be applied to coding subblock edges and transform block edges that coincide with subpicture boundaries where loop_filter_across_sub_pic_enabled_flag is equal to 0. Alternatively, the deblocking filter process is not applied to coding subblock edges and transform block edges that coincide with the upper or left boundary of a subpicture where loop_filter_across_sub_pic_enabled_flag is equal to 0. Alternatively, the deblocking filter process is not applied to coding subblock edges and transform block edges that coincide with subpicture boundaries where sub_pic_treated_as_pic_flag[i] is equal to 1 or 0. Alternatively, the deblocking filter process is not applied to coding subblock edges and transform block edges that coincide with the upper or left boundary of a subpicture. The clipping behavior may be specified to turn off SAO filtering across subpicture boundaries when loop_filter_across_sub_pic_enabled_flag for subpictures is equal to 0. The clipping behavior may also be specified to turn off ALF filtering across subpicture boundaries when loop_filter_across_sub_pic_enabled_flag for subpictures is equal to 0. loop_filter_across_tile_groups_enabled_flag may also be removed from the PPS. Therefore, when loop_filter_across_tiles_enabled_flag is equal to 0, in-loop filtering across tile group boundaries that are not subpicture boundaries is not turned off. Loop filtering behavior may include deblocking, SAO, and ALF.In another example, the clipping behavior is specified to turn off ALF filtering across tile boundaries when loop_filter_across_tiles_enabled_flag for a tile is equal to 0.

[0146] One or more of the above examples may be implemented as follows: A subpicture may be defined as a rectangular or square region of one or more tile groups or slices within a picture. The following divisions of the processing element, namely, division into components of each picture, division of each component into CTBs, division of each picture into subpictures, division of each subpicture into tile rows within the subpicture, division of each tile row within the subpicture into tiles, division of each tile row within the subpicture into tiles, and division of each subpicture into tile groups, may form spatial or component-based divisions.

[0147] The process for the CTB raster and tile scan process within the subpicture may be as follows: The list ColWidth[i], which specifies the width of the i-th tile column in CTB units, for i in the range of 0 to num_tile_columns_minus1, may be derived as follows:

number

[0148] The list RowHeight[j], which specifies the height of the j-th tile row in CTB units, for j in the range of 0 to num_tile_rows_minus1 (exclusive), is derived as follows:

number

[0149] The list ColBd[i], which specifies the position of the i-th tile column boundary in CTB units, for i in the range from 0 to num_tile_columns_minus1+1 (inclusive), is derived as follows:

number

[0150] The list RowBd[j], which specifies the position of the j-th tile row boundary in CTB units, for j in the range of 0 to num_tile_rows_minus1+1, is derived as follows:

number

[0151] The list CtbAddrRsToTs[ctbAddrRs], which specifies the conversion from the CTB address in the subpicture's CTB raster scan to the CTB address in the subpicture's tile scan, and contains ctbAddrRs in the range of 0 to SubPicSizeInCtbsY-1, is derived as follows:

number

[0152] The list CtbAddrTsToRs[ctbAddrTs], which specifies the conversion from the CTB address in a tile scan to the CTB address in the CTB raster scan of a subpicture, and contains ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1, is derived as follows:

number

[0153] The list TileId[ctbAddrTs], which specifies the conversion from CTB address to tile ID in subpicture tile scanning, and contains ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1, is derived as follows:

number

[0154] The list NumCtusInTile[tileIdx] for tileIdx in the range of 0 to NumTilesInSubPic - 1, which specifies the conversion from tile index to the number of CTUs in a tile, is derived as follows.

Number

[0155] The list FirstCtbAddrTs[tileIdx] for tileIdx in the range of 0 to NumTilesInSubPic - 1, which specifies the conversion from tile ID to the CTB address in the tile scan of the first CTB in a tile, is derived as follows.

Number

[0156] The value of ColumnWidthInLumaSamples[i] that specifies the width of the i-th tile column in units of luma samples is set equal to ColWidth[i] << CtbLog2SizeY for i in the range of 0 to num_tile_columns_minus1. The value of RowHeightInLumaSamples[j] that specifies the height of the j-th tile row in units of luma samples is set equal to RowHeight[j] << CtbLog2SizeY for j in the range of 0 to num_tile_rows_minus1.

[0157] The syntax of an exemplary sequence parameter set RBSP is as follows.

Table 1

[0158] The syntax of an exemplary picture parameter set RBSP is as follows. [Table 2] TIFF0007877629000014.tif34169

[0159] The syntax for an example general tile group header is as follows: [Table 3]

[0160] The syntax for an example coding tree unit is as follows: [Table 4]

[0161] The semantics of the exemplary sequence parameter set RBSP are as follows:

[0162] bit_depth_chroma_minus8 specifies the bit depth of the samples in the chroma sequence BitDepthC, and the value of the chroma quantization parameter range offset QpBdOffsetC is as follows:

number

[0163] The value of num_sub_pics_minus1 plus 1 specifies the number of subpictures in each coded picture within CVS. The value of num_sub_pics_minus1 must be in the range of 0 to 1024. The value of sub_pic_id_len_minus1 plus 1 specifies the number of bits used to represent the syntax element sub_pic_id[i] in SPS and the syntax element tile_group_sub_pic_id in the tile group header. The value of sub_pic_id_len_minus1 shall be in the range of Ceil(Log2(num_sub_pic_minus1+1)-1 or greater and 9 or less. sub_pic_level_present_flag is set to 1 to indicate that the syntax element sub_pic_level_idc[i] may exist. sub_pic_level_present_flag is set to 0 to indicate that the syntax element sub_pic_level_idc[i] does not exist. sub_pic_id[i] specifies the subpicture ID of the i-th subpicture of each coded picture in CVS. The length of sub_pic_id[i] is sub_pic_id_len_minus1+1 bits.

[0164] sub_pic_treated_as_pic_flag[i] is set to equal to 1 to specify that the i-th subpicture of each coded picture in the CVS is treated as a picture in the decoding process, except for the in-loop filtering operation. sub_pic_treated_as_pic_flag[i] is set to equal to 0 to specify that the i-th subpicture of each coded picture in the CVS is not treated as a picture in the decoding process, except for the in-loop filtering operation. sub_pic_level_idc[i] indicates the level to which the i-th subpicture sequence conforms, and the i-th subpicture sequence consists only of the VCL NAL units of the subpicture having a subpicture ID equal to sub_pic_id[i] in the CVS and the associated non-VCL NAL units. sub_pic_x_offset[i] specifies the horizontal offset of the upper-left corner luma sample of the i-th subpicture to the upper-left corner luma sample of each picture in the CVS, in units of luma samples. If it does not exist, the value of sub_pic_x_offset[i] is assumed to be equal to 0. sub_pic_y_offset[i] specifies the vertical offset of the top-left corner luma sample of the i-th subpicture relative to the top-left corner luma sample of each picture in CVS, in units of luma samples. If it does not exist, the value of sub_pic_y_offset[i] is assumed to be equal to 0. sub_pic_width_in_luma_samples[i] specifies the width of the i-th subpicture of each picture in CVS, in units of luma samples. If the sum of sub_pic_x_offset[i] and sub_pic_width_in_luma_samples[i] is less than pic_width_in_luma_samples, the value of sub_pic_width_in_luma_samples[i] is an integer multiple of CtbSizeY. If it does not exist, the value of sub_pic_width_in_luma_samples[i] is assumed to be equal to pic_width_in_luma_samples.sub_pic_height_in_luma_samples[i] specifies the height of the i-th subpicture of each picture in CVS, in units of luma samples. When the sum of sub_pic_y_offset[i] and sub_pic_height_in_luma_samples[i] is less than pic_height_in_luma_samples, the value of sub_pic_height_in_luma_samples[i] is an integer multiple of CtbSizeY. If it does not exist, the value of sub_pic_height_in_luma_samples[i] is assumed to be equal to pic_height_in_luma_samples.

[0165] The following constraints apply to bitstream compatibility: For any integer value of i or j, when i is equal to j, the values ​​of sub_pic_id[i] and sub_pic_id[j] shall not be the same. For any two subpictures subpicA and subpicB, when the subpicture ID of subpicA is smaller than the subpicture ID of subpicB, any coded tile group NAL unit of subPicA shall follow any coded tile group NAL unit of subPicB in the decoding order. The shape of a subpicture shall be such that each subpicture, when decoded, has an overall left boundary and an overall top boundary, which are composed of picture boundaries or boundaries of previously decoded subpictures.

[0166] The list SubPicIdx[spId] of spId values ​​equal to sub_pic_id[i] in the range i from 0 to num_sub_pics_minus1, which specifies the conversion from subpicture ID to subpicture index, is derived as follows:

number

[0167] log2_max_pic_order_cnt_lsb_minus4 specifies the value of the variable MaxPicOrderCntLsb, which is used in the decoding process for the picture order count, as follows:

number

[0168] The semantics of the exemplary picture parameter set RBSP are as follows:

[0169] When present, the values ​​of the PPS syntax elements pps_seq_parameter_set_id and loop_filter_across_tiles_enabled_flag are assumed to be the same for all PPS referenced by the tile group header of the coded picture. pps_pic_parameter_set_id identifies the PPS referenced by other syntax elements. The value of pps_pic_parameter_set_id must be in the range of 0 to 63. pps_seq_parameter_set_id specifies the value of sps_seq_parameter_set_id for active SPS. The value of pps_seq_parameter_set_id must be in the range of 0 to 15. loop_filter_across_sub_pic_enabled_flag is set to equal to 1 to specify that the loop filtering operation may be performed across the boundaries of subpictures referencing the PPS. The `loop_filter_across_sub_pic_enabled_flag` is set to 0 to specify that the in-loop filtering operation will not be performed across the boundaries of subpictures that reference the PPS.

[0170] `single_tile_in_sub_pic_flag` is set to 1 to specify that there is only one tile in each subpicture that references a PPS. `single_tile_in_sub_pic_flag` is set to 0 to specify that there are more than one tile in each subpicture that references a PPS. `num_tile_columns_minus1` plus 1 specifies the number of tile columns that separate the subpictures. `num_tile_columns_minus1` is between 0 and 1024. If it does not exist, the value of `num_tile_columns_minus1` is assumed to be equal to 0. `num_tile_rows_minus1` plus 1 specifies the number of tile rows that separate the subpictures. `num_tile_rows_minus1` is between 0 and 1024. If it does not exist, the value of `num_tile_rows_minus1` is assumed to be equal to 0. The variable NumTilesInSubPic is set to equal to (num_tile_columns_minus1+1)*(num_tile_rows_minus1+1). When single_tile_in_sub_pic_flag is equal to 0, NumTilesInSubPic is assumed to be greater than 1.

[0171] `uniform_tile_spacing_flag` is set to equal to 1 to specify that tile column boundaries and similarly tile row boundaries are uniformly distributed among subpictures. `uniform_tile_spacing_flag` is set to equal to 0 to specify that tile column boundaries and similarly tile row boundaries are not uniformly distributed among subpictures, but are explicitly signaled using the syntax elements `tile_column_width_minus1[i]` and `tile_row_height_minus1[i]`. When not present, the value of `uniform_tile_spacing_flag` is assumed to be equal to 1. `tile_column_width_minus1[i]` plus 1 specifies the width of the i-th tile column in CTB units. `tile_row_height_minus1[i]` plus 1 specifies the height of the i-th tile row in CTB units. `single_tile_per_tile_group` is set to equal to 1 to specify that each tile group referencing this PPS contains one tile. single_tile_per_tile_group is set to equal to 0 to specify that a tile group referencing this PPS may contain more than one tile.

[0172] rect_tile_group_flag is set to 0 to specify that the tiles within each tile group in a subpicture are in raster scan order and that tile group information is not signaled in PPS. rect_tile_group_flag is set to 1 to specify that the tiles within each tile group cover a rectangular or square area of ​​the subpicture and that tile group information is signaled in PPS. When single_tile_per_tile_group_flag is set to 1, rect_tile_group_flag is assumed to be equal to 1. num_tile_groups_in_sub_pic_minus1 plus 1 specifies the number of tile groups within each subpicture that reference PPS. The value of num_tile_groups_in_sub_pic_minus1 should be in the range of 0 or greater and NumTilesInSubPic-1 or less. If it does not exist and single_tile_per_tile_group_flag is equal to 1, the value of num_tile_groups_in_sub_pic_minus1 is presumed to be equal to NumTilesInSubPic-1.

[0173] `top_left_tile_idx[i]` specifies the tile index of the tile located in the top-left corner of the i-th tile group in the subpicture. The value of `top_left_tile_idx[i]` is assumed to be equal to the value of `top_left_tile_idx[j]` for any i that is not equal to j. If it does not exist, the value of `top_left_tile_idx[i]` is assumed to be equal to i. The length of the top_left_tile_idx[i] syntax element is Ceil(Log2(NumTilesInSubPic)) bits. bottom_right_tile_idx[i] specifies the tile index of the tile located in the bottom right corner of the i-th tile group of the subpicture. When single_tile_per_tile_group_flag is set to 1, bottom_right_tile_idx[i] is assumed to be equal to top_left_tile_idx[i]. The length of the bottom_right_tile_idx[i] syntax element is Ceil(Log2(NumTilesInSubPic)) bits.

[0174] A requirement for bitstream compatibility is that any particular tile must belong to only one tile group. The variable NumTilesInTileGroup[i], which specifies the number of tiles in the i-th tile group of a subpicture, and related variables are derived as follows:

number

[0175] The `loop_filter_across_tiles_enabled_flag` is set to 1 to specify that the in-loop filtering operation may be performed across tile boundaries in subpictures that reference PPS. The `loop_filter_across_tiles_enabled_flag` is set to 0 to specify that the in-loop filtering operation will not be performed across tile boundaries in subpictures that reference PPS. In-loop filtering operations include deblocking filters, sample-adaptive offset filters, and adaptive loop filtering operations. If not present, the value of `loop_filter_across_tiles_enabled_flag` is assumed to be 1. The value of num_ref_idx_default_active_minus1[i] plus 1 specifies an estimate of the variable NumRefIdxActive[0] for P or B tile groups with a num_ref_idx_active_override_flag equal to 0 when i is equal to 0, and specifies an estimate of NumRefIdxActive[1] for B tile groups with a num_ref_idx_active_override_flag equal to 0 when i is equal to 1. The value of num_ref_idx_default_active_minus1[i] shall be in the range of 0 to 14.

[0176] The semantics of an exemplary general tile group header are as follows: When present, the values ​​of the tile group header syntax elements tile_group_pic_order_cnt_lsb and tile_group_temporal_mvp_enabled_flag shall be the same in all tile group headers of the coded picture. When present, the value of tile_group_pic_parameter_set_id shall be the same in all tile group headers of the coded subpicture. tile_group_pic_parameter_set_id specifies the value of pps_pic_parameter_set_id for the PPS in use. The value of tile_group_pic_parameter_set_id shall be in the range of 0 to 63. A bitstream conformance requirement is that the value of the current picture's TemporalId shall be greater than or equal to the value of the TemporalId of each PPS referenced by the current picture's tile group. tile_group_sub_pic_id identifies the subpicture to which the tile group belongs. The length of tile_group_sub_pic_id is sub_pic_id_len_minus1+1 bits. The value of tile_group_sub_pic_id is assumed to be the same for all tile group headers of coded subpictures.

[0177] The variables SubPicWidthInCtbsY, SubPicHeightInCtbsY, and SubPicSizeInCtbsY are derived as follows:

number

[0178] The following variables, namely ColWidth[i], a list of i in the range of 0 to num_tile_columns_minus1, specifying the width of the i-th tile column in CTB units; RowHeight[j], a list of j in the range of 0 to num_tile_rows_minus1, specifying the height of the j-th tile row in CTB units; ColBd[i], a list of i in the range of 0 to num_tile_columns_minus1+1, specifying the position of the boundary of the i-th tile column in CTB units; and the j-th tile column in CTB units. RowBd[j] is a list of j values ​​in the range of 0 to num_tile_rows_minus1+1 that specifies the position of the tile row boundary, CtbAddrRsToTs[ctbAddrRs] is a list of ctbAddrRs values ​​in the range of 0 to SubPicSizeInCtbsY-1 that specifies the conversion from the CTB address in the subpicture's CTB raster scan to the CTB address in the subpicture's tile scan, A list of ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1, CtbAddrTsToRs[ctbAddrTs], which specifies the conversion to CTB addresses; a list of ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1, TileId[ctbAddrTs], which specifies the conversion from CTB addresses to tile IDs in subpicture tile scanning; a list of ctbAddrTs in the range of 0 to SubPicSizeInCtbsY-1, TileId[ctbAddrTs], which specifies the conversion from tile index to the number of CTUs in the tile, NumTilesInSubPic-1, which is between 0 and 1. A list of tileIdx ranges NumCtusInTile[tileIdx], which specifies the conversion from tile ID to the CTB address in the tile scan of the first CTB within the tile, a list of tileIdx ranges from 0 to NumTilesInSubPic-1, FirstCtbAddrTs[tileIdx], which specifies the width of the i-th tile column in units of luma samples, a list of i ranges from 0 to num_tile_columns_minus1, ColumnWidthInLumaSamples[i],The list RowHeightInLumaSamples[j], which specifies the height of the j-th tile row in units of Luma Samples, for j in the range of 0 to num_tile_rows_minus1, is derived by calling the CTB raster and tile scan conversion process.

[0179] The values ​​of ColumnWidthInLumaSamples[i] for i in the range of 0 to num_tile_columns_minus1 and RowHeightInLumaSamples[j] for j in the range of 0 to num_tile_rows_minus1 are all greater than 0. The variables SubPicLeftBoundaryPos, SubPicTopBoundaryPos, SubPicRightBoundaryPos, and SubPicBotBoundaryPos are derived as follows.

number

[0180] For each tile in the current subpicture with index i = 0.NumTilesInSubPic - 1, the variables TileLeftBoundaryPos[i], TileTopBoundaryPos[i], TileRightBoundaryPos[i], and TileBotBoundaryPos[i] are derived as follows:

number

[0181] `tile_group_address` specifies the tile address of the first tile in the tile group. If it does not exist, the value of `tile_group_address` is assumed to be equal to 0. If `rect_tile_group_flag` is equal to 0, then the following applies: The tile address is the tile ID, the length of `tile_group_address` is Ceil(Log2(NumTilesInSubPic)) bits, and the value of `tile_group_address` is in the range of 0 or greater and less than or equal to NumTilesInSubPic-1. Otherwise (if `rect_tile_group_flag` is equal to 1), then the following applies: The tile address is the tile group index of the tile group within the tile group in the subpicture, the length of `tile_group_address` is Ceil(Log2(num_tile_groups_in_sub_pic_minus1+1)) bits, and the value of `tile_group_address` is in the range of 0 or greater and less than or equal to num_tile_groups_in_sub_pic_minus1.

[0182] The following constraints must be met for bitstream compatibility to be valid: The value of tile_group_address must not be equal to the value of tile_group_address of any other coded tile group NAL unit in the same coded subpicture. The tile groups of a subpicture must be in increasing order of these tile_group_address values. The shape of a tile group in a subpicture must be such that each tile, when decoded, has an overall left boundary and an overall top boundary that are composed of the subpicture boundary or the boundary of a previously decoded tile.

[0183] num_tiles_in_tile_group_minus1, if it exists, specifies the number of tiles in the tile group minus 1. The value of num_tiles_in_tile_group_minus1 is assumed to be in the range of 0 or greater and less than or equal to NumTilesInSubPic-1. If it does not exist, the value of num_tiles_in_tile_group_minus1 is assumed to be equal to 0. The variable NumTilesInCurrTileGroup, which specifies the number of tiles in the current tile group, and the variable TgTileIdx[i], which specifies the tile index of the i-th tile in the current tile group, are derived as follows.

number

[0184] An exemplary derivation process for predicting time-based motion vectors is as follows: The variables mvLXCol and availableFlagLXCol are derived as follows: If tile_group_temporal_mvp_enabled_flag is equal to 0, both components of mvLXCol are set to equal to 0, and availableFlagLXCol is set to equal to 0. Otherwise (if tile_group_temporal_mvp_enabled_flag is equal to 1), the following steps apply: The motion vectors for the same position in the lower right, as well as the sample positions at the bottom and right boundaries, are derived as follows:

number

[0185] An exemplary luma sample bilinear interpolation process is as follows: The luma position in all sample units (xInti, yInti) is derived for i=0..1 as follows: When sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following holds:

number

number

[0186] An exemplary Luma sample 8-tap interpolation filtering process is as follows. The Luma position in all sample units (xInti, yInti) is derived for i=0..7 as follows. When sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, the following holds:

number

number

[0187] An exemplary chroma sample interpolation process is as follows: The variable xOffset is set to equal (sps_ref_wraparound_offset_minus1+1)*MinCbSizeY) / SubWidthC. The chroma position in all sample units (xInti, yInti) is derived for i=0..3 as follows:

[0188] If sub_pic_treated_as_pic_flag[SubPicIdx[tile_group_subpic_id]] is equal to 1, then the following applies:

number

number

[0189] An exemplary deblocking filter process is as follows: The deblocking filter process applies to all coding subblock edges and transform block edges of a picture, except for the following types of edges: edges at the boundary of a picture, edges that coincide with the boundary of a subpicture where loop_filter_across_sub_pic_enabled_flag is equal to 0, edges that coincide with the boundary of a tile where loop_filter_across_tiles_enabled_flag is equal to 0, edges that coincide with the upper or left boundary of a tile group or within a tile group where tile_group_deblocking_filter_disabled_flag is equal to 1, edges that do not correspond to the 8x8 sample grid boundary of the component under consideration, edges in chroma components where both sides of the edge use interpretation, edges of chroma transform blocks that are not edges of the associated transform unit, and edges that cross chroma transform blocks of coding units with an IntraSubPartitionsSplit value that is not equal to ISP_NO_SPLIT.

[0190] An exemplary deblocking filter process in one direction is as follows. For each coding unit having a coding block width log2CbW, a coding block height log2CbH, and the position (xCb, yCb) of the top-left sample of the coding block, when edgeType is equal to EDGE_VER and xCb % 8 is equal to 0, or when edgeType is equal to EDGE_HOR and yCb % 8 is equal to 0, the edge is filtered by the steps in the following order. The coding block width nCbW is set equal to 1 << log2CbW, and the coding block height nCbH is set equal to 1 << log2CbH. The variable filterEdgeFlag is derived as follows. When edgeType is equal to EDGE_VER and one or more of the following conditions are true, filterEdgeFlag is set equal to 0. The left boundary of the current coding block is the left boundary of the picture. The left boundary of the current coding block is the left or right boundary of a subpicture and loop_filter_across_sub_pic_enabled_flag is equal to 0. The left boundary of the current coding block is the left boundary of a tile and loop_filter_across_tiles_enabled_flag is equal to 0. Otherwise, when edgeType is equal to EDGE_HOR and one or more of the following conditions are true, the variable filterEdgeFlag is set equal to 0. The top boundary of the current luma coding block is the top boundary of the picture. The top boundary of the current coding block is the top or bottom boundary of a subpicture and loop_filter_across_sub_pic_enabled_flag is equal to 0. The top boundary of the current coding block is the top boundary of a tile and loop_filter_across_tiles_enabled_flag is equal to 0. Otherwise, filterEdgeFlag is set equal to 1.

[0191] An exemplary CTB modification process is as follows: Depending on the values ​​of pcm_loop_filter_disabled_flag, pcm_flag[xYi][yYj] and cu_transquant_bypass_flag for the coding unit containing the coding block covering recPicture[xSi][ySj], the following applies to all sample positions (xSi,ySj) and (xYi,yYj) at i=0..nCtbSw-1 and j=0..nCtbSh-1. For all sample positions (xSik',ySjk') and (xYik',yYjk') at k=0..1, if one or more of the following conditions are true, edgeIdx is set to equal to 0: The sample at position (xSik',ySjk') is outside the picture boundary. The sample at position (xSik', ySjk') belongs to a different subpicture, and the loop_filter_across_sub_pic_enabled_flag for the tile group to which the sample recPicture[xSi][ySj] belongs is equal to 0. The loop_filter_across_tiles_enabled_flag is equal to 0, and the sample at position (xSik', ySjk') belongs to a different tile.

[0192] The following is the filtering process for an exemplary coding tree block regarding Luma samples. To derive the reproduced Luma sample alfPictureL[x][y] after filtering, each reproduced Luma sample in the current Luma coding tree block recPictureL[x][y] is filtered by x,y=0..CtbSizeY-1 as follows. The position (hx,vy) of each corresponding Luma sample (x,y) in the given array of Luma samples recPicture is derived as follows. If loop_filter_across_tiles_enabled_flag for tile A containing the Luma sample at position (hx,vy) is equal to 0, then the following holds true, where tileIdx is the tile index of tile A.

number

number

number

[0193] An exemplary derivation process for the ALF transpose and filter index for luma samples is as follows: The position (hx,vy) of each corresponding luma sample (x,y) in a given array recPicture of luma samples is derived as follows: If loop_filter_across_tiles_enabled_flag for tile A containing the luma sample at position (hx,vy) is equal to 0, then the following holds true, where tileIdx is the tile index of tile A.

number

number

number

[0194] The following is an example of the filtering process for a coding tree block regarding chroma samples. To derive the reproduced chroma sample alfPicture[x][y] after filtering, each reproduced chroma sample in the current chroma coding tree block recPicture[x][y] is filtered by x,y=0..ctbSizeC-1 as follows. The position (hx,vy) of each corresponding chroma sample (x,y) in the given array of chroma samples recPicture is derived as follows. If loop_filter_across_tiles_enabled_flag for tile A containing the chroma sample at position (hx,vy) is equal to 0, then the following holds true, where tileIdx is the tile index of tile A.

number

number

number

[0195] The sum of the variables is derived as follows:

number

number

[0196] Figure 12 is a schematic diagram of an exemplary video coding device 1200. The video coding device 1200 is suitable for implementing the examples / embodiments disclosed herein as described herein. The video coding device 1200 includes a downstream port 1220, an upstream port 1250, and / or a transceiver unit (Tx / Rx) 1210, including transmitters and / or receivers for communicating data upstream and / or downstream over a network. The video coding device 1200 also includes a processor 1230, including a logic unit and / or a central processing unit (CPU) for processing data, and a memory 1232 for storing data. The video coding device 1200 may also include electrical, optical-to-electrical (OE) components, electrical-to-optical (EO) components, and / or wireless communication components coupled to the upstream port 1250 and / or downstream port 1220 for communicating data over an electrical, optical, or wireless communication network. The video coding device 1200 may also include an input and / or output (I / O) device 1260 for communicating data to and from the user. The I / O device 1260 may include output devices such as a display for displaying video data and a speaker for outputting audio data. The I / O device 1260 may also include input devices such as a keyboard, mouse, trackball, and / or corresponding interfaces for interacting with such output devices.

[0197] The processor 1230 is implemented by hardware and software. The processor 1230 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and digital signal processors (DSPs). The processor 1230 communicates with downstream port 1220, Tx / Rx 1210, upstream port 1250, and memory 1232. The processor 1230 includes a coding module 1214. The coding module 1214 implements embodiments disclosed herein, such as methods 100, 1300, and 1400, which may use current blocks 801 and / or 901 that can be coded according to one-way inter prediction 600 and / or two-way inter prediction 700 based on a candidate list generated according to an in-loop filter 1000, a bitstream 1100, a picture 500, and / or pattern 900. The coding module 1214 may also implement any other method / mechanism described herein. Furthermore, the coding module 1214 may implement a codec system 200, an encoder 300, and / or a decoder 400. For example, the coding module 1214 can implement the implementations of the first, second, third, fourth, fifth, and / or sixth examples as described above. Thus, the coding module 1214 provides the video coding device 1200 with additional functionality and / or coding efficiency when coding video data. Thus, the coding module 1214 improves the functionality of the video coding device 1200 and addresses problems specific to video coding technology. Furthermore, the coding module 1214 results in the conversion of the video coding device 1200 to different states.Alternatively, the coding module 1214 can be implemented as instructions stored in memory 1232 and executed by the processor 1230 (for example, as a computer program product stored on a non-temporary medium).

[0198] Memory 1232 includes one or more memory types such as disks, tape drives, solid-state drives, read-only memory (ROM), random access memory (RAM), flash memory, ternary content-addressable memory (TCAM), and static random-access memory (SRAM). Memory 1232 may also be used as an overflow data storage device to store such programs when they are selected for execution and to store instructions and data read during program execution.

[0199] Figure 13 is a flowchart of an exemplary method 1300 for encoding a video sequence of subpictures, such as subpicture 510 of picture 500, into a bitstream such as bitstream 1100. Method 1300 may be used by an encoder such as a codec system 200, encoder 300, and / or video coding device 1200, when performing method 100 for encoding the current blocks 801 and / or 901 according to one-way interpretation 600 and / or two-way interpretation 700 on a candidate list generated according to pattern 900 by using an in-loop filter 1000.

[0200] Method 1300 may begin when an encoder receives a video sequence containing multiple pictures and decides to encode the video sequence into a bitstream, for example, based on user input. In step 1301, the encoder divides the picture into subpictures. In step 1303, the encoder determines the width, height, and offset of the subpictures.

[0201] In step 1305, the encoder encodes the subpicture width, subpicture height, and subpicture offset into a bitstream. The subpicture width, height, and offset may also be encoded into the bitstream in SPS. The subpicture width and subpicture height are encoded in units of CTB. A CTB contains a predetermined number of samples, and therefore a CTB describes a predetermined unit size in the picture and / or subpicture. In some examples, the subpicture width is encoded into the bitstream as subpic_width_minus1, where subpic_width_minus1 is one less CTB than the number of CTBs in the subpicture width. Furthermore, the subpicture height may also be encoded into the bitstream as subpic_height_minus1, where subpic_height_minus1 is one less CTB than the number of CTBs in the subpicture height. Thus, the subpicture width or height can be omitted from the bitstream and inferred if the width or height each consists of one CTB. This saves bits and therefore increases coding efficiency. Furthermore, signaling the width and height in a -1 format may also reduce the coding size of the width and height in the bitstream by one bit in some cases, thus increasing coding efficiency for this reason as well. The subpicture offset may be specified as the vertical position and horizontal position of the top-left CTU of the subpicture. For example, the subpicture offset may be specified as the vertical and horizontal difference between the top-left CTU of the picture and the top-left CTU of the subpicture. The vertical and horizontal positions of the top CTUs may also be signaled in units of CTB. In some examples, the vertical position of the top-left CTU of the subpicture may be encoded in the bitstream as subpic_ctu_top_left_y. Furthermore, the vertical position of the top-left CTU of the subpicture may be encoded in the bitstream as subpic_ctu_top_left_x.

[0202] In step 1307, the encoder encodes the coding blocks of the subpicture into a bitstream. Furthermore, in step 1309, the encoder stores the bitstream for communication with the decoder.

[0203] Figure 14 is a flowchart of an exemplary method 1400 for decoding a video sequence containing subpictures, such as subpicture 510 of picture 500, from a bitstream such as bitstream 1100. Method 1400 may be used by a decoder such as a codec system 200, decoder 400, and / or video coding device 1200 when performing method 100 for decoding the current blocks 801 and / or 901 according to one-way interpretation 600 and / or two-way interpretation 700 on a candidate list generated according to pattern 900 by using an in-loop filter 1000.

[0204] Method 1400 may begin when the decoder begins receiving a bitstream of coded data representing a video sequence, for example, as a result of Method 1300. In step 1401, the decoder receives a bitstream containing / divided into subpictures.

[0205] In step 1403, the decoder obtains the width, height, and offset of the subpicture from the bitstream. The width, height, and offset of the subpicture may be obtained from the SPS in the bitstream. The width and height of the subpicture are included in the bitstream in units of CTB. A CTB contains a predetermined number of samples, and therefore a CTB describes a predetermined unit size in the picture and / or subpicture. In some examples, the width of the subpicture is obtained from the bitstream in the variable subpic_width_minus1, where subpic_width_minus1 is one less CTB than the number of CTBs in the subpicture width. Furthermore, the height of the subpicture may be obtained from the bitstream in the variable subpic_height_minus1, where subpic_height_minus1 is one less CTB than the number of CTBs in the subpicture height. Thus, the width or height of the subpicture can be omitted from the bitstream and inferred if the width or height is one CTB each. This saves bits and therefore increases coding efficiency. Furthermore, signaling the width and height in a -1 format may also reduce the coding size of the width and height in the bitstream by one bit in some cases, and therefore also increases coding efficiency for this reason. The offset of a subpicture may be specified as the vertical position of the top-left CTU of the subpicture and the horizontal position of the top-left CTU of the subpicture. For example, the offset of a subpicture may be specified in the bitstream as the vertical and horizontal difference between the top-left CTU of the picture and the top-left CTU of the subpicture. The vertical and horizontal positions of the top CTUs may also be signaled in units of CTB. In some examples, the vertical position of the top-left CTU of the subpicture may be obtained from the bitstream in the variable subpic_ctu_top_left_y. Furthermore, the vertical position of the top-left CTU of the subpicture may be obtained from the bitstream in the variable subpic_ctu_top_left_x.

[0206] In step 1405, the decoder decodes the coding blocks of the subpicture based on the subpicture's width, height, and offset. For example, the decoder can determine the size and position of the subpicture based on its width, height, and offset. The decoder can also use this information to position the subpicture at a specified location (e.g., relative to the picture), include the appropriate slices within the subpicture, and include the correct coding blocks in the slices before decoding the blocks to reproduce the picture and / or subpicture. In step 1407, the decoder can transfer the decoded coding blocks of the subpicture for display as part of the decoded video sequence.

[0207] Figure 15 is a schematic diagram of an exemplary system 1500 for coding a video sequence of images such as picture 500, which includes subpictures such as subpicture 510 in a bitstream such as bitstream 1100. System 1500 may be implemented by a codec system 200, an encoder 300, a decoder 400, and / or a video coding device such as video coding device 1200. Furthermore, system 1500 may be used to implement methods 100, 1300, and / or 1400 for coding the current blocks 801 and / or 901 by using an in-loop filter 1000 and / or by one-way inter-prediction 600 and / or two-way inter-prediction 700 to a candidate list generated according to pattern 900.

[0208] System 1500 includes a video encoder 1502. The video encoder 1502 includes a partitioning module 1505 for partitioning a picture into subpictures. The video encoder 1502 further includes a determination module 1506 for determining the width and height of the subpictures. The video encoder 1502 further includes an encoding module 1507 for encoding the width and height of the subpictures into a bitstream in units of CTB. Furthermore, the encoding module 1507 is for encoding the coding blocks of the subpictures into a bitstream. The video encoder 1502 further includes a storage module 1508 for storing the bitstream for communication to the decoder. The video encoder 1502 further includes a transmission module 1509 for transmitting the bitstream to the video decoder 1510. The video encoder 1502 may be further configured to perform any of the steps of method 1300.

[0209] System 1500 also includes a video decoder 1510. The video decoder 1510 includes a receive module 1511 for receiving a bitstream containing a picture containing subpictures. The video decoder 1510 further includes an acquire module 1512 for obtaining the width of the subpictures in units of CTB and the height of the subpictures in units of CTB from the bitstream. The video decoder 1510 further includes a decode module 1513 for decoding the coding blocks of the subpictures based on the width and height of the subpictures. The video decoder 1510 further includes a transfer module 1515 for transferring the coding blocks of the subpictures for display as part of the decoded video sequence. The video decoder 1510 may be further configured to perform any of the steps of Method 1400.

[0210] The first component is directly coupled to the second component when there are no intervening components other than a line, trace, or other medium between the first component and the second component. The first component is indirectly coupled to the second component when there are intervening components other than a line, trace, or other medium between the first component and the second component. The term "coupled" and its variations include both being directly coupled and being indirectly coupled. The use of the term "about" means a range that includes ±10% of the subsequent number unless otherwise specified.

[0211] It should also be understood that the steps of the exemplary methods described herein need not necessarily be performed in the order described, and that such order of steps is to be regarded as merely exemplary. Similarly, additional steps may be included in such methods, and specific steps may be omitted or combined in methods consistent with various embodiments of the present disclosure.

[0212] Although several embodiments are provided in the present disclosure, it can be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. This example is to be considered illustrative and not restrictive, and its intention is not to be limited to the details given herein. For example, various elements or components may be coupled or integrated into other systems, or certain features may be omitted or not implemented.

[0213] Furthermore, the technologies, systems, subsystems, and methods described and illustrated individually or separately in various embodiments may be combined or integrated with other systems, components, technologies, or methods without departing from the scope of the present disclosure. Other examples of modifications, substitutions, and changes can be verified by those skilled in the art and may be made without departing from the spirit and scope disclosed herein.

Claims

1. A method implemented in a decoder, Steps include receiving a bitstream containing coded data of a picture including subpictures, wherein the subpictures may contain more than one slice, each slice may contain more than one tile, and each tile is a rectangular area of ​​a coding tree unit (CTU), and The steps include obtaining the width of the subpicture in units of a coding tree block (CTB) and the height of the subpicture in units of a CTB from the bitstream, The steps include obtaining the subpicture identifier (ID) of the subpicture from the bitstream, A step of decoding the coding block of the subpicture based on the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture. A method that includes this.

2. The method according to claim 1, further comprising the step of obtaining a flag for the subpicture from the bitstream, wherein a flag equal to 1 specifies that the subpicture is treated as a picture in the decoding process except for an in-loop filtering operation, and a flag equal to 0 specifies that the subpicture is not treated as a picture in the decoding process except for an in-loop filtering operation.

3. The method according to claim 2, wherein the flag is indicated as sub_pic_treated_as_pic_flag.

4. The method according to any one of claims 1 to 3, wherein the width of the subpicture is stored in the bitstream as subpic_width_minus1, and the height of the subpicture is stored in the bitstream as subpic_height_minus1.

5. The method according to any one of claims 1 to 4, wherein the width, height and subpicture ID of the subpicture are obtained from the sequence parameter set (SPS) in the bitstream.

6. A method implemented in an encoder, The steps include dividing a picture into subpictures, wherein each subpicture may contain more than one slice, each slice may contain more than one tile, and each tile is a rectangular area of ​​a coding tree unit (CTU), and The steps include determining the width and height of the subpicture, The steps include encoding the width and height of the subpicture into a bitstream in units of coding tree blocks (CTB), The steps include encoding the subpicture identifier (ID) of the subpicture into the bitstream, The steps include encoding the coding block of the subpicture into the bitstream. A method that includes this.

7. The method according to claim 6, further comprising the step of encoding a flag for the subpicture into the bitstream, the flag being set to equal to 1 to specify that the subpicture is to be treated as a picture in the decoding process except for an in-loop filtering operation, and the flag being set to equal to 0 to specify that the subpicture is not to be treated as a picture in the decoding process except for an in-loop filtering operation.

8. The method according to claim 6 or 7, wherein the width of the subpicture is encoded in the bitstream as subpic_width_minus1, and the height of the subpicture is encoded in the bitstream as subpic_height_minus1.

9. The method according to any one of claims 6 to 8, wherein the width, height and subpicture ID of the subpicture are encoded in the sequence parameter set (SPS) of the bitstream.

10. A video decoding device, A video decoding device comprising a processor, a receiver coupled to the processor, a memory coupled to the processor, and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform the method of any one of claims 1 to 5.

11. A video encoding device, A video encoding device comprising a processor, a receiver coupled to the processor, a memory coupled to the processor, and a transmitter coupled to the processor, wherein the processor, receiver, memory, and transmitter are configured to perform the method of any one of claims 6 to 9.

12. A non-temporary computer-readable medium that includes a computer-executable instruction, when executed by a processor, causing the processor to perform the method according to any one of claims 1 to 5.

13. A non-temporary computer-readable medium that includes a computer-executable instruction, when executed by a processor, causing the processor to perform the method according to any one of claims 6 to 9.

14. It is a decoder, A receiving unit configured to receive a bitstream containing coded data of a picture including subpictures, wherein the subpictures may contain more than one slice, each slice may contain more than one tile, and each tile is a rectangular area of ​​a coding tree unit (CTU), and An acquisition unit configured to obtain the width of the subpicture in units of coding tree blocks (CTB) and the height of the subpicture in units of CTB from the bitstream, and an acquisition unit further configured to obtain the subpicture identifier (ID) of the subpicture from the bitstream, A decoding unit configured to decode the coding block of a subpicture based on the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture. A decoder that includes this.

15. The decoder according to claim 14, wherein the decoder is further configured to perform the method described in any one of claims 2 to 5.

16. It is an encoder, A partitioning unit configured to divide a picture into subpictures, wherein each subpicture may contain more than one slice, each slice may contain more than one tile, and each tile is a rectangular area of ​​a coding tree unit (CTU), and A determination unit configured to determine the width and height of the subpicture, An encoding unit configured to encode the width and height of a subpicture into a bitstream in units of coding tree blocks (CTBs), encode the subpicture identifier (ID) of the subpicture into the bitstream, and encode the coding block of the subpicture into the bitstream. An encoder that includes this.

17. The encoder according to claim 16, wherein the encoder is further configured to perform the method described in any one of claims 7 to 9.

18. A computer program, when executed on a computer or processor, includes program code for performing the method described in any one of claims 1 to 5.

19. A computer program, when executed on a computer or processor, includes program code for performing the method described in any one of claims 6 to 9.

20. A device for storing and decoding a video bitstream, The system includes a communication interface, a processor, and a storage medium, wherein the communication interface is configured to receive and / or transmit a bitstream, the storage medium is configured to store the bitstream, the bitstream includes coded data of a picture including subpictures, the width of the subpictures in units of a coding tree block (CTB), the height of the subpictures in units of a CTB, and a subpicture identifier (ID) of the subpictures, the subpictures may include more than one slice, each slice may include more than one tile, and each tile is a rectangular area of ​​a coding tree unit (CTU). A device wherein the processor analyzes the bitstream to obtain the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture, and the processor further decodes the coding block of the subpicture based on the width of the subpicture, the height of the subpicture, and the subpicture ID of the subpicture.

Images