Video decoding / encoding method and bit stream transmission method

Abstract

The invention provides a video decoding / encoding method and a bitstream transmission method. A method and an apparatus for acquiring information on subunits split from a screen are disclosed. According to an embodiment of the present invention, a method for acquiring information on subunits split from a screen is provided. The method comprises the steps of: decoding, from a bitstream, CTU size information indicating a size of a coding tree unit (CTU) within a picture; decoding, from a bitstream, sub-picture splitting information indicating sub-pictures within the picture in units of the CTU size; decoding splitting information of one or more tiles within the picture from a bitstream; and decoding splitting information of one or more slices within the picture from a bitstream.

Description

[0001] This application is a divisional application of the original application No. 202080082074.8 (International Application No. PCT / KR2020 / 012769, filed on September 22, 2020, entitled "Method and apparatus for acquiring information on sub-units split from a picture"). TECHNICAL FIELD

[0002] The present disclosure relates to video encoding and decoding, and more particularly, to a method and apparatus for improving encoding and decoding efficiency by efficiently obtaining information on various sub-units. BACKGROUND

[0003] Since the volume of video data is larger than that of voice data or still image data, storing or transmitting video data without compression processing requires a large amount of hardware resources including a memory.

[0004] Therefore, when storing or transmitting video data, an encoder is generally used to compress video data for storage or transmission. Then, a decoder receives the compressed video data and decompresses and reproduces the video data. Techniques for such video compression include H.264 / AVC and High Efficiency Video Coding (HEVC), which improves encoding efficiency by about 40% compared to H.264 / AVC.

[0005] However, video size, resolution, and frame rate are gradually increasing, and thus the amount of data to be encoded is also increasing. Therefore, there is a need for a new compression technique that has better encoding efficiency than existing compression techniques and has high image quality. SUMMARY

[0006] TECHNICAL PROBLEM

[0007] An object of the present disclosure is to provide an improved video encoding and decoding technique to meet this need. In particular, one aspect of the present disclosure relates to a technique for improving encoding and decoding efficiency by various methods of splitting a picture into sub-units.

[0008] TECHNICAL SOLUTION

[0009] According to one aspect, the present disclosure provides a method for acquiring information on sub-units split from a picture, the method including: decoding, from a bitstream, CTU size information indicating a size of a coding tree unit (CTU) within a picture; decoding, from the bitstream, sub-picture split information representing a sub-picture within the picture in units of the CTU size; decoding, from the bitstream, split information on splitting the picture into one or more tiles; and decoding, from the bitstream, split information on splitting the picture into one or more slices.

[0010] According to another aspect, the present application provides a decoder configured to decode, from a bitstream, information on sub-units into which a picture is partitioned; and a picture reconstructor configured to reconstruct the picture based on the information on the sub-units, wherein the decoder is configured to decode, from the bitstream, CTU size information indicating a size of a coding tree unit (CTU) within the picture, to decode, from the bitstream, sub-picture partitioning information representing sub-pictures within the picture in units of the CTU size, to decode, from the bitstream, partitioning information on partitioning the picture into one or more tiles; and to decode, from the bitstream, partitioning information on partitioning the picture into one or more slices.

[0011] Technical Effects

[0012] As described above, according to one embodiment of the present disclosure, since overlap of syntax elements for partitioning a picture into sub-units is prevented, bit efficiency can be improved.

[0013] Further, according to another embodiment of the present disclosure, whether to apply in-loop filtering between sub-unit boundaries can be reasonably designed based on partitioning relationships between sub-units.

[0014] Further, according to another embodiment of the present disclosure, since partitioning information of a sub-picture is represented in units of a CTU, uniformity with other sub-units by which partitioning information of a sub-picture can be efficiently represented can be ensured. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a schematic block diagram of a video encoding apparatus capable of implementing the technology of the present application.

[0016] Figure 2 An exemplary block partitioning structure using QTBTTT structure is shown.

[0017] Figure 3 Exemplary intra prediction modes are shown.

[0018] Figure 4 is a schematic block diagram of a video decoding apparatus capable of implementing the technology of the present application.

[0019] Figures 5 to 8 Slices, tiles, and sub-pictures are shown.

[0020] Figure 9 is a flowchart showing one example of a method capable of implementing the technology of the present disclosure.

[0021] Figure 10 is a flowchart showing one example of a method for partitioning a picture into tiles.

[0022] Figure 11 is a flowchart illustrating one example of a method for partitioning a picture into blocks.

[0023] Figure 12 The partitioning relationship between tiles / slices is exemplified.

[0024] Figure 13 is a flowchart illustrating one example of a method for partitioning a picture into slices.

[0025] Figures 14 to 17 is a flowchart illustrating various examples of a method for obtaining information about whether to apply filtering of a slice boundary.

[0026] Figure 18 is a flowchart illustrating one example of a method for partitioning a picture into sub-pictures.

[0027] Figure 19 is a flowchart illustrating a specific method for partitioning a picture into sub-pictures. DETAILED DESCRIPTION

[0028] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that, when adding reference numerals to constituent elements in the respective drawings, the same reference numerals denote the same elements although these elements are illustrated in different drawings. Also, in the following description of the present disclosure, detailed description of known functions and configurations incorporated herein will be omitted in order not to obscure the subject matter of the present disclosure.

[0029] Figure 1 is a schematic block diagram of a video encoding apparatus capable of implementing the technology of the present invention. Hereinafter, the video encoding apparatus and the elements of the apparatus will be described with reference to Figure 1 The video encoding apparatus and the elements of the apparatus will be described with reference to

[0030] The video encoding apparatus includes a block splitter 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, an encoder 150, an inverse quantizer 160, an inverse transformer 165, an adder 170, a filter unit 180, and a memory 190.

[0031] The respective elements of the video encoding apparatus can be implemented in hardware or software or a combination of hardware and software. The functions of the respective elements can be implemented in software, and a microprocessor can be implemented to perform the software functions corresponding to the respective elements.

[0032] A video is composed of a plurality of pictures. Each picture is split into a plurality of regions, and encoding is performed on each region. For example, a picture is split into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is split into one or more coding tree units (CTUs). Each CTU is split into one or more coding units (CUs) through a tree structure. Information applied to each CU is coded as syntax of the CU, and information applied to a plurality of CUs included in a common CTU is coded as syntax of the CTU. In addition, information applied to all blocks in a common tile is coded as syntax of the tile or as syntax of a tile group, which is a set of a plurality of tiles, and information applied to all blocks constituting a picture is coded in a picture parameter set (PPS) or a picture header. In addition, information commonly referred to by a plurality of pictures is coded in a sequence parameter set (SPS). In addition, information referred to by one or more common SPSs is coded in a video parameter set (VPS).

[0033] The block splitter 110 determines the size of a coding tree unit (CTU). Information about the CTU size (CTU size) is coded as syntax of the SPS or the PPS and is transmitted to the video decoding apparatus.

[0034] The block splitter 110 splits each picture constituting a video into a plurality of CTUs having a predetermined size, and then recursively splits the CTUs using a tree structure. In the tree structure, a leaf node serves as a coding unit (CU), which is a basic unit of encoding.

[0035] The tree structure can be a quad tree (QT) in which a node (or a parent node) is split into 4 sub (or child) nodes having the same size, a binary tree (BT) in which a node is split into 2 child nodes, a ternary tree (TT) in which a node is split into 3 child nodes in a ratio of 1:2:1, or a structure formed by a combination of two or more of the QT structure, the BT structure, and the TT structure. For example, a QTBT (quad tree plus binary tree) structure or a QTBTTT (quad tree plus binary tree ternary tree) structure can be used. Herein, BTTT can be collectively referred to as a multi-type tree (MTT).

[0036] Figure 2 A QTBTTT split tree structure is illustrated. As Figure 2As shown, a CTU can be initially split in a QT structure. QT splitting can be repeated until the size of the split block reaches the minimum block size of the leaf node allowed in QT (minqtsize). The encoder 150 encodes and signals to the video decoding device a first flag (QT_split_flag) indicating whether each node of the QT structure is split into 4 nodes of the lower level. When the leaf node of the QT is not larger than the maximum block size of the root node allowed in BT (maxbtsize), it can be further split into one or more of a BT structure or a TT structure. In the BT structure and / or the TT structure, there can be multiple splitting directions. For example, there can be two directions, i.e., horizontal splitting and vertical splitting of the node block. As shown, when the MTT splitting starts, the encoder 150 encodes and signals to the video decoding device a second flag (mtt_split_flag) indicating whether the node is split, a flag indicating the splitting direction (vertical or horizontal), and / or a flag indicating the splitting type (binary or ternary). Figure 2

[0037] As another example of tree structure, when a block is split using a QTBTTT structure, the encoder 150 encodes and signals to the video decoding device information about a CU split flag (split_cu_flag) indicating that the block has been split and a QT split flag (split_qt_flag) indicating whether the splitting type is QT splitting. When the value of split_cu_flag indicates that the block is not split, the block of the node becomes a leaf node in the split tree structure and is used as a coding unit (CU), which is the basic unit of encoding. When the value of split_cu_flag indicates that the block is split, whether the splitting type is QT or MTT is distinguished by the value of split_qt_flag. When the splitting type is QT, there is no additional information. When the splitting type is MTT, a flag (mtt_split_cu_vertical_flag) indicating the MTT splitting direction (vertical or horizontal) and / or a flag (mtt_split_cu_binary_flag) indicating the MTT splitting type (binary or ternary) are encoded by the encoder 150 and signaled to the video decoding device.

[0038] ​As another example of a tree structure, when QTBT is used, there can be two types of splitting, i.e., splitting a node's block horizontally (i.e., symmetric horizontal splitting) and splitting vertically (i.e., symmetric vertical splitting) into two blocks of the same size. The encoder 150 encodes and sends to the video decoding device split flags that indicate whether each node of the BT structure is split into blocks of a lower level and split type information that indicates the type of split. There can be additional types that split a node's block into two asymmetric blocks. The asymmetric split types can include a type that splits a block into two rectangular blocks with a size ratio of 1 :3, and a type that splits a block of a node diagonally.

[0039] A CU can have various sizes depending on the QTBT or QTBTTT splitting of a CTU. Hereinafter, a block corresponding to a CU to be encoded or decoded (i.e., a leaf node of the QTBTTT) is referred to as a "current block."

[0040] The predictor 120 predicts the current block to generate a prediction block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0041] In general, each current block in a picture can be predictively coded. The prediction of a current block can be performed using an intra prediction technique, which is based on data from the picture containing the current block, or an inter prediction technique, which is based on data from pictures that are coded before the picture containing the current block. Inter prediction includes uni-prediction and bi-prediction.

[0042] The intra predictor 122 predicts pixels in the current block using pixels (reference pixels) located around the current block in the current picture containing the current block. There are a plurality of intra prediction modes according to the prediction direction. For example, as shown in Figure 3 the plurality of intra prediction modes can include non-directional modes including a planar mode and a DC mode, and 65 directional modes. For each prediction mode, neighboring pixels and an algorithm to be used are defined differently.

[0043] The intra predictor 122 can determine an intra prediction mode to be used for encoding the current block. In some examples, the intra predictor 122 can encode the current block using a plurality of intra prediction modes and select an appropriate intra prediction mode to use from among the tested modes. For example, the intra predictor 122 can use rate-distortion analysis of a plurality of tested intra prediction modes to calculate rate-distortion values, and can select an intra prediction mode having the best rate-distortion characteristics among the tested modes.

[0044] The intra predictor 122 selects one of a plurality of intra prediction modes and predicts the current block using neighboring pixels (reference pixels) and an equation determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the encoder 150 and transmitted to the video decoding apparatus.

[0045] The inter predictor 124 generates a prediction block for the current block through a motion compensation process. The inter predictor searches for a block most similar to the current block in a reference picture that is earlier in encoding and decoding than the current picture, and generates a prediction block for the current block based on the searched block. Then, the inter predictor generates a motion vector corresponding to a displacement between the current block in the current picture and the prediction block in the reference picture. Generally, motion estimation is performed on a luminance component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The encoder 150 encodes and transmits motion information including information about a reference picture used to predict the current block and information about the motion vector to the video decoding apparatus.

[0046] The subtracter 130 generates a residual block by subtracting the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0047] The transformer 140 transforms a residual signal in the residual block having pixel values in a spatial domain into a transform coefficient in a frequency domain. The transformer 140 can transform the residual signal in the residual block using the total size of the current block as a transform unit. Alternatively, the transformer can split the residual block into sub-blocks of a transform region and a non-transform region, and transform the residual signal using only the sub-blocks of the transform region as a transform unit. Here, the transform region sub-block can be one of two rectangular blocks having a size ratio of 1:1 based on a horizontal axis (or a vertical axis). In this case, a flag (cu_sbt_flag) indicating that only the sub-block has been transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are additionally encoded by the encoder 150 and signaled to the video decoding apparatus. Further, the size of the transform region sub-block can have a size ratio of 1:3 based on the horizontal axis (or the vertical axis). In this case, a flag (cu_sbt_quad_flag) for distinguishing the split is additionally encoded by the encoder 150 and signaled to the video decoding apparatus.

[0048] The quantizer 145 quantizes the transform coefficient output from the transformer 140 and outputs the quantized transform coefficient to the encoder 150.

[0049] The encoder 150 generates a bitstream by encoding the quantized transform coefficients using an encoding method such as context-based adaptive binary arithmetic coding (CABAC). The encoder 150 encodes information related to block splitting, such as CTU size, CU split flags, QT split flags, MTT split directions, and MTT split types, so that a video decoding device splits blocks in the same manner as the video encoding device.

[0050] In addition, the encoder 150 encodes information about a prediction type indicating whether the current block is encoded by intra prediction or by inter prediction, and encodes either intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (information about a reference picture and a motion vector) according to the prediction type.

[0051] The inverse quantizer 160 inverse-quantizes the quantized transform coefficients output from the quantizer 145 to generate transform coefficients. The inverse transformer 165 transforms the transform coefficients output from the inverse quantizer 160 from the frequency domain to the spatial domain and reconstructs a residual block.

[0052] The adder 170 adds the reconstructed residual block to the prediction block generated by the predictor 120 to reconstruct the current block. Pixels in the reconstructed current block are used as reference pixels for intra prediction for a next block.

[0053] The filter unit 180 filters the reconstructed pixels to reduce blockiness, ringing artifacts, and blurring artifacts resulting from block-based prediction and transform / quantization. The filter unit 180 can include a deblocking filter 182 and a sample adaptive offset (SAO) filter 184.

[0054] The deblocking filter 182 filters boundaries between reconstructed blocks to remove blockiness caused by block-based encoding / decoding, and the SAO filter 184 further filters the deblocking-filtered video. The SAO filter 184 is a filter for compensating for differences between reconstructed pixels and original pixels caused by lossy encoding.

[0055] The reconstructed block filtered by the deblocking filter 182 and the SAO filter 184 is stored in the memory 190. When all blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of blocks in a next picture to be encoded.

[0056] Meanwhile, the picture encoder divides a picture into sub-units and encodes the picture in units of CUs included in the sub-units. The picture encoder can include the remaining components included in the video encoding apparatus except for the encoder 150. The encoder 150 encodes information about the sub-units divided from the picture and signals the encoded information to the video decoding apparatus. The method of encoding information about the sub-units by the encoder 150 will be described in detail below.

[0057] Figure 4 is a schematic functional block diagram of a video decoding apparatus capable of implementing the technology of the present application. Hereinafter, the video decoding apparatus and elements of the apparatus will be described with reference to Figure 4

[0058] The video decoding apparatus can include a decoder 410, an inverse quantizer 420, an inverse transformer 430, a predictor 440, an adder 450, a filter unit 460, and a memory 470.

[0059] Like the video encoding apparatus of Figure 1 Each element of the video decoding apparatus can be implemented as hardware or software, or as a combination of hardware and software. Further, the functions of each element can be implemented as software, and a microprocessor can be implemented to perform the functions of the software corresponding to each element.

[0060] The decoder 410 determines a current block to be decoded by decoding a bitstream received from the video encoding apparatus and extracting information related to block splitting, and extracts prediction information and information about a residual signal required to reconstruct the current block.

[0061] The decoder 410 extracts information about a CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), determines the size of the CTU, and splits a picture into CTUs of the determined size. Then, the decoder determines the CTU as the uppermost layer, i.e., the root node of the tree structure, and extracts splitting information about the CTU to split the CTU using the tree structure.

[0062] For example, when the CTU is split using the QTBTTT structure, a first flag (QT_split_flag) related to QT splitting is first extracted, and each node is split into 4 nodes of a lower layer. Then, for a node corresponding to a leaf node of the QT, a second flag (MTT_split_flag) related to MTT splitting and information about a splitting direction (vertical / horizontal) and / or a splitting type (binary / trinary) are extracted, and the leaf node is split in the MTT structure. In this way, each node under the leaf node of the QT is recursively split in the BT structure or the TT structure.

[0063] ​As another example, when a CTU is split using a QTBT structure, a CU split flag (split_cu_flag) indicating whether a CU is split or not is first extracted. If the corresponding block is split, a QT split flag (split_qt_flag) is extracted. When the split type is not QT but MTT, a flag (mtt_split_cu_vertical_flag) indicating the MTT split direction (vertical or horizontal) and / or a flag (mtt_split_cu_binary_flag) indicating the MTT split type (binary or ternary) are additionally extracted. In the splitting process, each node can be subjected to zero or more recursive QT splits and then zero or more recursive MTT splits. For example, a CTU can be immediately MTT split, or can be only multiple QT splits.

[0064] As another example, when a CTU is split using a QTBT structure, a first flag (QT_split_flag) is related to QT split, and each node is split into 4 nodes of a lower layer. For a node corresponding to a leaf node of QT, a split_flag indicating whether the node is further BT split or not and split direction information are extracted.

[0065] In determining a current block to be decoded by tree structure split, the decoder 410 extracts information on prediction type indicating whether the current block is intra predicted or inter predicted. When the prediction type information indicates intra prediction, the decoder 410 extracts a syntax element for intra prediction information (intra prediction mode) of the current block. When the prediction type information indicates inter prediction, the decoder 410 extracts a syntax element for inter prediction information, i.e., information indicating a motion vector and a reference picture referred by the motion vector.

[0066] The decoder 410 extracts information on quantized transform coefficients of the current block as information on a residual signal.

[0067] The inverse quantizer 420 inverse quantizes the quantized transform coefficients and inverse transforms the inverse quantized transform coefficients from a frequency domain to a spatial domain to reconstruct a residual signal, thereby generating a residual block for the current block.

[0068] Further, when the inverse transformer 430 inversely transforms only a partial region (sub-block) of the transform block, a flag indicating that only the sub-block of the transform block has been transformed (cu_sbt_flag) is extracted, as well as direction information (vertical / horizontal) (cu_sbt_horizontal_flag) and / or sub-block position information (cu_sbt_pos_flag) about the sub-block. Then, the residual signal is reconstructed by inversely transforming the transform coefficients of the sub-block from the frequency domain to the spatial domain. For the region that has not been inversely transformed, the residual signal is padded with "0". Thereby, the final residual block for the current block is created.

[0069] The predictor 440 can include an intra predictor 442 and an inter predictor 444. The intra predictor 442 is activated when the prediction type of the current block is intra prediction, and the inter predictor 444 is activated when the prediction type of the current block is inter prediction.

[0070] The intra predictor 442 determines an intra prediction mode of the current block among a plurality of intra prediction modes based on syntax elements for the intra prediction mode extracted from the decoder 410, and predicts the current block based on reference pixels around the current block according to the intra prediction mode.

[0071] The inter predictor 444 determines a motion vector of the current block and a reference picture referred by the motion vector based on syntax elements for the inter prediction information extracted from the decoder 410, and predicts the current block based on the motion vector and the reference picture.

[0072] The adder 450 reconstructs the current block by adding the residual block output from the inverse transformer to the prediction block output from the inter predictor or the intra predictor. Pixels in the reconstructed current block are used as reference pixels for intra prediction of blocks to be decoded later.

[0073] The filter unit 460 can include a deblocking filter 462 and an SAO filter 464. The deblocking filter 462 performs deblocking filtering on boundaries between reconstructed blocks to remove block artifacts caused by block-based decoding. The SAO filter 464 performs additional filtering on the reconstructed blocks after deblocking filtering to compensate for differences between reconstructed pixels and original pixels caused by lossy encoding. The reconstructed blocks filtered by the deblocking filter 462 and the SAO filter 464 are stored in the memory 470. When all blocks in a picture are reconstructed, the reconstructed picture is used as a reference picture for inter prediction of blocks in a picture to be encoded later.

[0074] Meanwhile, the decoder 410 obtains information about sub-units partitioned from a picture by decoding a bitstream. The picture reconstructor 4000 reconstructs a picture based on the obtained information about the sub-units. The picture reconstructor 4000 can include the remaining components in the video decoding apparatus except for the decoder 410. The method of obtaining information about the sub-units by the decoder 410 will be described in detail below.

[0075] In the HEVC international standard, there are two types of methods of partitioning a picture (image or screen) for the purpose of parallel processing. One is a slice partitioning method, and the other is a tile partitioning method.

[0076] The slice partitioning method provides a function of partitioning a picture into CTUs and encoding / decoding the CTUs in a raster scan order. A plurality of slice segments can exist in one slice segment, and there are two types of segments. The types of slice segments include an independent slice segment and a dependent slice segment.

[0077] The independent slice segment is a slice segment that does not depend on another slice when performing intra prediction, inter prediction, encoding mode, and entropy encoding. The independent slice segment has header information. The dependent slice segment is a slice segment that depends on another slice (refers to another previous independent slice segment) when performing intra prediction, inter prediction, encoding mode, and entropy encoding. The dependent slice segment refers to header information of a previous independent slice segment for most of the header information of the dependent slice segment. In other words, only a part of the header information of the dependent slice segment is signaled.

[0078] Figure 5 FIG. 1(a) shows two slices partitioned from a picture, and the two slices can be separated by a slice boundary. One slice can include one independent slice segment and zero or more dependent slice segments. The first slice in the upper portion of the picture has one independent slice segment and two dependent slice segments separated by a dashed line. The second slice in the lower portion of the picture has only one independent slice segment (includes zero dependent slice segments).

[0079] Table 1 and Table 2 show syntax structures for the slice partitioning method.

[0080] [Table 1]

[0081]

[0082] [Table 2]

[0083]

[0084] Table 1 shows a flag (dependent_slice_segments_enabled_flag) indicating whether to allow the use of dependent slice segments. The flag can be defined by a PPS. Table 2 shows slice segment header information.

[0085] As shown in Table 2, a flag (first_slice_segment_in_pic_flag) indicating whether a specific slice segment is the first slice segment in a picture is encoded and signaled. The first slice segment becomes an independent slice segment. For a segment other than the first slice segment, a flag (dependent_slice_segment_flag) indicating whether the segment is a dependent slice segment is encoded and signaled. In addition, an address (slice_segment_address) of a related slice segment is signaled together. Here, the slice_segment_address indicates an index value of the first CTU in the related slice segment.

[0086] 2) The slice partitioning method provides a function of encoding / decoding a picture by segmenting / partitioning the picture into a plurality of columns and rows based on CTUs. The picture can be partitioned into tiles based on a uniform partitioning scheme or a separate (non-uniform) partitioning scheme.

[0087] Each partitioned tile does not depend on other tiles during intra prediction and entropy coding, and the dependency on in-loop filtering can be controlled by a flag defined in a PPS. There is no separate header for a tile.

[0088] Figure 5 (b) of FIG. 1 shows that a picture is partitioned into 3x3 tiles, and each tile can be separated by tile boundaries (column boundaries and row boundaries). A plurality of tiles can be encoded / decoded in a raster scan order, and a plurality of CTUs in one tile can also be encoded / decoded in a raster scan order. Numbers shown in each CTU indicate an order of performing encoding / decoding.

[0089] Table 3 shows a syntax structure for the tile partitioning method.

[0090] [Table 3]

[0091]

[0092] As shown in Table 3, a flag (tiles_enabled_flag) indicating the on / off of tile partitioning is defined and signaled in a PPS. When the related flag is on, syntax elements specifying tile sizes are additionally signaled.

[0093] The syntax elements that specify the tile size can include num_tile_columns_minus1, num_tile_rows_minus1, and uniform_spacing_flag, num_tile_columns_minus1 indicates a value obtained by subtracting 1 from the number of tiles (the number of tile columns) that are divided in the vertical direction, num_tile_rows_minus1 indicates a value obtained by subtracting 1 from the number of tiles (the number of tile rows) that are divided in the horizontal direction, and uniform_spacing_flag indicates whether the picture is uniformly divided in the horizontal and vertical directions.

[0094] When the picture is not uniformly divided (uniform_spacing_flag = off), tile column width information (column_width_minus1) and tile row height information (row_height_minus1) are additionally signaled. In addition, a flag (loop_filter_across_tiles_enabled_flag) indicating whether to apply in-loop filtering at the boundary between tiles is signaled.

[0095] Figure 6 The difference between the above-described slice partitioning method and the tile partitioning method is shown. Figure 6 (a) of FIG. 1 shows slices (a total of 4 slices) partitioned according to the slice partitioning method, and Figure 6 (b) of FIG. 1 shows tiles (a total of 4 tiles) partitioned according to the tile partitioning method. Each number indicates the number of CTUs.

[0096] Only horizontal boundaries can exist between the partitioned slices. In contrast, horizontal and vertical boundaries can exist between the partitioned tiles.

[0097] The disclosure proposes a method for obtaining information about sub-units partitioned from a picture.

[0098] The sub-units (sub-blocks or sub-areas) can include one or more of sub-pictures, slices, tiles / squares, and CTUs. The information about the sub-units can include an enabling flag of the sub-units and partitioning information.

[0099] The enabling flag is a syntax element indicating whether to perform in-loop filtering at the boundary between the sub-units. The partitioning information of the sub-units corresponds to information for partitioning the picture into various sub-units. Accordingly, "obtaining the partitioning information of the sub-units" can be "partitioning the picture into the sub-units".

[0100] This disclosure proposes a screen segmentation method based on sub-unit segmentation information for various purposes. Specifically, this disclosure proposes 1) a sub-screen segmentation method for synthesizing different sequences, 2) a slice segmentation method for bitstream packets, and 3) a block / square segmentation method for parallel processing.

[0101] Tiles / blocks, slices and sub-images

[0102] A tile / block refers to a frame region obtained by dividing the frame into sub-regions for parallel processing. Tiles / blocks are processed independently during intra-frame prediction and entropy coding. In other words, tiles / blocks do not share reference samples during intra-frame prediction and do not share context during entropy coding. Whether to apply intra-loop filtering to the boundaries between tiles / blocks can be controlled by a separate flag.

[0103] A slice represents a data packet unit that configures an independent Network Abstraction Layer (NAL) bitstream. Multiple pieces / blocks can be included in a single slice, or multiple slices can be included in a single piece / block.

[0104] Slices can have two types or modes. The first mode is the rectangular slice mode, in which tiles / squares with similar characteristics or arbitrary tiles / squares are designated as a region, while the second mode is the non-rectangular slice (raster scan slice) mode, in which tiles / squares are designated as a region at a specific location, and the encoding / decoding of tiles / squares is performed in raster scan order.

[0105] Figure 7 Examples of rectangular slice mode and raster scan slice mode are shown. Figure 7 In the diagram, areas separated by dashed lines represent CTUs, areas separated by straight lines represent tiles / blocks, and areas separated by dotted lines represent slices.

[0106] like Figure 7 As shown in (a), a frame can be divided into a total of 216 (18×12) CTUs, a total of 24 (6×4) tiles / blocks, and a total of 9 rectangular slices. Figure 7 As shown in (b), a frame can be divided into a total of 216 (18×12) CTUs, a total of 12 (3×4) tiles / blocks, and a total of 3 raster scan slices.

[0107] A sub-picture is an area designated for composition with other content. A sub-picture may include at least one slice.

[0108] Figure 8An example of compositing different contents in units of sub-pictures is shown. The area filled with a diagonal line in the upper right direction represents a picture in content 1. The area filled with a horizontal line pattern represents a picture in content 2. The area filled with a vertical line pattern represents a picture in content 3.

[0109] As shown on the left side of FIG. 1, Figure 8 Each picture can be composed of 4 slices, and a bitstream can be generated for each slice unit. Further, each slice can be defined as a sub-picture. The bitstream physically generated in units of slices is input to a bitstream extractor and a merger, and undergoes extraction and compositing processes. Here, the logical unit of the extraction and compositing processes is a sub-picture. As a result, as shown on the right side of FIG. 1, a new picture is generated from the compositing, in which the upper left portion is content 1, the lower left portion is content 2, and the right portion is content 3. Figure 8

[0110] The bitstream extractor and the merger extract the bitstreams generated in units of slices from different contents, and composites the extracted bitstreams in units of sub-pictures. To this end, the pictures of each content are divided into sub-pictures. In other words, a sub-picture is an area including at least one slice for content compositing.

[0111] Hereinafter, a method of obtaining information about a sub-unit will be described. Figure 9 A flowchart illustrating one embodiment of the disclosure is shown.

[0112] A video encoding apparatus can divide a picture into various sub-units. After being divided into a plurality of CTUs having a predetermined size, the picture can be divided into sub-pictures, tiles / slices, by using the CTU size as a basic unit.

[0113] For example, the picture can be divided into one or more sub-pictures based on the CTU size as a basic unit, and the picture can be divided into one or more tiles / slices based on the CTU size as a basic unit. In the case of slice division, the picture can be divided into one or more slices based on the tile / slice as a basic unit. Here, since the tile / slice is divided based on the CTU size as a basic unit, the slice is divided based on the CTU size as a basic unit.

[0114] Accordingly, information about sub-picture division (sub-picture division information) can represent sub-pictures within a picture in units of the CTU size, information about tile / slice division (tile / slice division information) can represent tiles / slices within a picture in units of the CTU size, and information about slice division (slice division information) can represent slices within a picture in units of the tile / slice (size unit of the CTU).

[0115] ​The information on the CTU size (CTU size information), the sub-picture partitioning information, the tile / slab partitioning information, and the slice partitioning information are encoded and signaled to the video decoding apparatus. The CTU size information is defined and signaled at the SPS level or the PPS level, the sub-picture partitioning information is defined and signaled at the SPS level, and the tile / slab partitioning information and the slice partitioning information are defined and signaled at the PPS level.

[0116] Further, the video encoding apparatus determines whether to apply in-loop filtering on the boundaries between the sub-units, encodes an enabling flag indicating whether to apply the in-loop filtering, and signals the encoded enabling flag to the video decoding apparatus.

[0117] The enabling flag indicating whether to apply in-loop filtering on the boundaries between the sub-units is defined and signaled at the SPS level; the enabling flag indicating whether to apply in-loop filtering on the boundaries between the tiles / slabs and the enabling flag indicating whether to apply in-loop filtering on the boundaries between the slices are defined and signaled at the PPS level.

[0118] The decoder 410 decodes the partitioning information of the sub-units from the bitstream (S910). The CTU size information, the sub-picture partitioning information, the tile / slab partitioning information, and the slice partitioning information included in the partitioning information of the sub-units are obtained by decoding from the bitstream. The decoder 410 partitions the picture into the sub-units based on the obtained partitioning information (S930). Specifically, after being partitioned into the CTUs having the CTU size indicated by the CTU size information, the picture can be partitioned into the sub-pictures, the tiles / slabs, and the slices based on the sub-picture partitioning information, the tile / slab partitioning information, and the slice partitioning information.

[0119] The decoder 410 decodes the enabling flag indicating whether to apply in-loop filtering on the boundaries between the sub-units from the bitstream (S920). The filter unit 460 determines whether to apply in-loop filtering on the boundaries between the sub-units based on the enabling flag (S930).

[0120] Embodiment 1

[0121] Embodiment 1 discloses a method for obtaining tile partitioning information and a method for obtaining slab partitioning information. In other words, Embodiment 1 relates to a method for partitioning a picture into tiles and a method for partitioning a picture into slabs.

[0122] Table 4 shows one example of syntax structure related to Embodiment 1.

[0123] [Table 4]

[0124]

[0125] Embodiment 1-1

[0126] Embodiment 1-1 relates to a method for partitioning a picture into tiles (a method for obtaining tile partitioning information). Figure 10 An example of Embodiment 1-1 is shown.

[0127] The video coding device determines whether there is only one tile within the picture and sets the determination result as the value of the single_tile_in_pic_flag syntax element. When single_tile_in_pic_flag == 1, it indicates that there is only one tile in the picture, and when single_tile_in_pic_flag == 0, it indicates that there is one or more tiles in the picture.

[0128] In the case where there is one or more tiles within the picture (single_tile_in_pic_flag == 0), the video coding device determines whether the corresponding one or more tiles are uniformly partitioned and sets the determination result as the value of the uniform_tile_spacing_flag syntax element. When uniform_tile_spacing_flag == 1, it indicates that the tiles are uniformly partitioned from the picture, and when uniform_tile_spacing_flag == 0, it indicates that the tiles are not uniformly partitioned from the picture.

[0129] When the tiles are uniformly partitioned (uniform_tile_spacing_flag == 1), the video coding device determines the width of the tile columns and the height of the tile rows and sets the determination results as the value of the tile_cols_width_minus1 syntax element and the value of the tile_rows_height_minus1 syntax element, respectively. tile_cols_width_minus1 is a syntax element carrying the width of the tile columns in CTU size, except for the width of the rightmost tile column of the picture. tile_rows_height_minus1 is a syntax element carrying the height of the tile rows in CTU size, except for the height of the bottom tile row of the picture.

[0130] When the tiles are not uniformly spaced (uniform tile spacing flag == 0), the video encoding device determines the number of tile columns and the number of tile rows, and sets the determination results as the value of the num tile columns minusl syntax element and the value of the num tile rows minusl syntax element, respectively. The num tile columns minusl is a syntax element carrying a value obtained by subtracting 1 from the number of tile columns, and the num tile rows minusl is a syntax element carrying a value obtained by subtracting 1 from the number of tile rows.

[0131] The video encoding device determines the width of the non-uniformly spaced tile columns and the height of the non-uniformly spaced tile rows, and sets the determination results as the value of the tile column width minusl[i] syntax element and the value of the tile row height minusl[i] syntax element, respectively. The tile column width minusl[i] is a syntax element indicating the width of the non-uniformly spaced tile columns in CTU size units. The syntax element is not defined for the width of the rightmost tile column of the picture. The tile row height minusl[i] is a syntax element indicating the height of the non-uniformly spaced tile rows. The syntax element is not defined for the height of the tile row at the bottom of the picture.

[0132] The video encoding device encodes and signals the single tile in pic flag, the uniform tile spacing flag, the tile cols width minusl, the tile rows height minusl, the num tile columns minusl, the num tile rows minusl, the tile column width minusl[i], and the tile row height minusl[i] to the video decoding device.

[0133] The decoder 410 decodes the single tile in pic flag from the bitstream (S1010), and determines whether there is only one tile in the picture or there is one or more tiles in the picture based on the value of the single tile in pic flag (S1020).

[0134] When there is only one tile in the picture (single_tile_in_pic_flag = = 1), the decoder 410 considers the picture as one tile and terminates the tile partitioning process. On the other hand, when there is one or more tiles in the picture (single_tile_in_pic_flag = = 0), the decoder 410 decodes uniform_tile_spacing_flag from the bitstream (S1030).

[0135] The decoder 410 determines whether the tiles in the picture are uniformly partitioned based on the decoded uniform_tile_spacing_flag value (S1040). When the tiles are uniformly partitioned (uniform_tile_spacing_flag = = 1), the decoder 410 decodes tile_cols_width_minusl and tile_rows_height_minusl from the bitstream (S1050). On the other hand, when the tiles are not uniformly partitioned (uniform_tile_spacing_flag = = 0), the decoder 410 decodes num_tile_columns_minusl and num_tile_rows_minusl from the bitstream (S1060), and decodes tile_column_width_minusl [i] and tile_row_height_minusl [i] from the bitstream for the number indicated by num_tile_columns_minusl and num_tile_rows_minusl, respectively (S1070).

[0136] The decoder 410 can partition the picture into non-uniform tiles based on the values of num_tile_columns_minusl, num_tile_rows_minusl, tile_column_width_minusl [i], and tile_row_height_minusl [i], or uniformly partition the picture into tiles based on the values of tile_cols_width_minusl and tile_rows_height_minusl.

[0137] Embodiment 1-2

[0138] Embodiment 1-2 relates to a method for partitioning a picture into tiles (a method for obtaining tile partitioning information). The picture is allowed to be partitioned into tiles only in the horizontal direction. Figure 11 One example of Embodiment 1-2 is shown.

[0139] The video encoding device can determine whether a brick within a picture is split into multiple tiles and set the determination result as the value of a brick_splitting_present_flag syntax element. When brick_splitting_present_flag == 1, it indicates that one or more bricks in the picture are split into multiple tiles, and when brick_splitting_present_flag == 0, it indicates that there is no brick in the picture that is split into multiple tiles. The brick_splitting_present_flag is coded and signaled to the video decoding device.

[0140] The video encoding device can determine the number of bricks in a picture and set the determination result as the value of a num_tiles_in_pic_minusl syntax element. When the bricks in the picture are uniformly split (uniform_tile_spacing_flag == 1) and one or more bricks in the picture are split into multiple tiles (brick_splitting_present_flag == 1), the num_tile_in_pic_minusl is coded and signaled to the video decoding device.

[0141] The video encoding device checks the height of the i-th brick (the number of CTU rows in the i-th brick, RowHeight[[i]) and sets the values of syntax elements brick_split_flag[i], uniform_brick_spacing_flag[i], brick_height_minusl[i], num_brick_rows_minus2[i] and brick_row_height_minusl[i][j] based on the result.

[0142] brick_split_flag[i] is a syntax element indicating whether the i-th brick is split into tiles. When the value of RowHeight[i] exceeds 1, brick_split_flag[i] can be set to "1", which is a value indicating that the i-th brick is split into tiles and is coded and signaled to the video decoding device. This is because when the value of RowHeight[i] exceeds 1, tile splitting in the horizontal direction is possible. On the other hand, when the value of RowHeight[i] is 1 or less, brick_split_flag[i] is set to "0" and is not signaled to the video decoding device, which is a value indicating that the i-th brick is not split into tiles. This is because when the value of RowHeight[i] is 1 or less, it is not possible to split tiles in the horizontal direction.

[0143] uniform_brick_spacing_flag[i] is a syntax element indicating whether the i-th brick is uniformly split into bricks. When the value of rowheight[i] exceeds 2, both uniform split and non-uniform split are possible, thus, uniform_brick_spacing_flag[i] can be set to a value of "1" indicating uniform split, or set to a value of "0" indicating non-uniform split. In this case, uniform_brick_spacing_flag[i] can be coded and signaled to the video decoding device. On the other hand, when the value of rowheight[i] is 2 or less, only 1:1 uniform split is possible, thus, uniform_brick_spacing_flag[i] is set to "1" and not signaled to the video decoding device.

[0144] brick_height_minusl[i] is a syntax element indicating a value obtained by subtracting 1 from the height of a uniformly split brick row in units of CTU size. The syntax element is not defined for the height of the bottom brick row within the i-th brick. When bricks are uniformly split, brick_height_minusl[i] is coded and signaled to the video decoding device in the case where the value of rowheight[i] exceeds 3. In contrast, when bricks are uniformly split and the value of rowheight[i] is 3, only 1:1:1 uniform split is possible, thus, brick_height_minusl[i] is set to 0 (brick_height_minusl[i] == 0) and not signaled to the video decoding device.

[0145] num_brick_rows_minus2[i] is a syntax element indicating a value obtained by subtracting 2 from the number of non-uniformly split bricks. When bricks are non-uniformly split and the value of RowHeight[i] exceeds 3, num_brick_rows_minus2[i] is coded and signaled to the video decoding device. On the other hand, when bricks are non-uniformly split and the value of RowHeight[i] is 3, only 1:2 or 2:1 non-uniform split is possible, thus, num_brick_rows_minus2[i] is set to 0 (num_brick_rows_minus2[i] == 0), and not signaled to the video decoding device.

[0146] brick_row_height_minusl[i][j] is a syntax element that indicates, in units of CTU size, a value obtained by subtracting 1 from the height of the j-th tile row in the non-uniformly split tile row. For example, when the number of non-uniformly split tiles is 3 (num_brick_rows_minus2[i] == 1), brick_row_height_minusl[i][j] is set to a value obtained by subtracting 1 from each of the heights of two tiles. In another example, when the number of non-uniformly split tiles is 5 (num_brick_rows_minus2[i] == 3), brick_row_height_minusl[i][j] is set to a value obtained by subtracting 1 from each of the heights of 4 tiles. brick_row_height_minusl[i][j] is encoded and signaled to the video coding device.

[0147] The decoder 410 decodes brick_splitting_present_flag from the bitstream (S1110) and determines the value of the decoded brick_splitting_present_flag (S1120).

[0148] When the tiles within the picture are uniformly split (uniform_tile_spacing_flag == 1) and one or more tiles within the picture are split into multiple bricks (brick_splitting_present_flag == 1), the decoder 410 decodes num_tile_in_pic_minusl from the bitstream (S1130). When the tiles within the picture are not uniformly split (uniform_tile_spacing_flag == 0) or when no tiles within the picture are split into multiple bricks (brick_splitting_present_flag == 0), num_tile_in_pic_minusl is not signaled and the value of num_tile_in_pic_minusl is set or derived to a value obtained by subtracting 1 from the number of tiles within the picture.

[0149] The decoder 410 checks the number of CTU rows in the i-th brick (RowHeight[i]) (S1140), and when RowHeight[i] > 1, decodes brick_split_flag from the bitstream (S1150). When the value of RowHeight[i] is 1 or less, brick_split_flag is not signaled, and the value of brick_split_flag is derived or set to 0.

[0150] When brick_split_flag == 0, the decoder 410 terminates brick splitting (S1160), but when brick_split_flag == 1, the decoder 410 determines the value of RowHeight[i] (S1170). When the value of RowHeight[i] exceeds 2, uniform_brick_spacing_flag is decoded from the bitstream (S1180), but when the value of RowHeight[i] is 2 or less, uniform_brick_spacing_flag is not decoded, and the value of uniform_brick_spacing_flag is set or derived to 1.

[0151] The decoder 410 determines whether the bricks are uniformly partitioned based on the value of uniform_brick_spacing_flag (S1182), and determines the value of RowHeight[i] (S1184, S1190). When uniform_brick_spacing_flag == 1 and when the value of RowHeight[i] exceeds 3, the decoder 410 decodes brick_height_minus1[i] from the bitstream (S1186). On the other hand, when uniform_brick_spacing_flag == 1 and the value of RowHeight[i] does not exceed 3, brick_height_minus1[i] is not decoded, and the value of brick_height_minus1[i] is set or derived to be 0. When uniform_brick_spacing_flag == 0 and the value of RowHeight[i] exceeds 3, the decoder 410 decodes num_brick_rows_minus2[i] from the bitstream (S1192). Meanwhile, when uniform_brick_spacing_flag == 0 and the value of RowHeight[i] does not exceed 3, num_brick_rows_minus2[i] is not decoded, and the value of num_brick_rows_minus2[i] is set or derived to be 0. The decoder 410 decodes brick_row_height_minus1[i][j] from the bitstream (S1194).

[0152] As described above, the brick partitioning method proposed in the present disclosure is configured to signal and decode brick_height_minus1[i] and num_brick_rows_minus2[i] only when the value of RowHeight[i] exceeds 3. On the other hand, in the conventional brick partitioning method, brick_height_minus1[i] and num_brick_rows_minus2[i] are signaled and decoded for both the case when the value of RowHeight[i] exceeds 3 and the case when the value of RowHeight[i] does not exceed 3 (in other words, regardless of whether the value of RowHeight[i] exceeds 3 or not). Therefore, the brick partitioning method of the present disclosure can improve the bit efficiency compared to the conventional brick partitioning method.

[0153] Embodiment 2

[0154] Embodiment 2 relates to a method for partitioning a picture into slices (a method for obtaining slice partitioning information) and a method for obtaining an enabling flag.

[0155] Table 5 shows the syntax structure related to Embodiment 2.

[0156] [Table 5]

[0157]

[0158] Embodiment 2-1

[0159] Embodiment 2-1 relates to a method for partitioning a picture into slices. The slice partitioning can be implemented in the form of including one or more bricks / tiles. Figure 12 The partitioning relationship between the bricks / tiles and the slices is shown, Figure 13 The method for partitioning a picture into slices is shown.

[0160] Figure 12 (a) of FIG. 1 shows a total of 9 bricks / tiles (3x3) partitioned from a picture, and Figure 12 (b) to (d) of FIG. 1 show the partitioning relationship between the bricks / tiles (solid line) and the slices (dotted line) based on the partitioning of the bricks / tiles of (a). As shown in (b) of FIG. 1, one brick / tile can include one slice (1:1 partitioning relationship), and multiple bricks / tiles can include one rectangular slice, as shown in (c) of FIG. 1 (n:1 partitioning relationship, where n is a natural number greater than or equal to 2). Further, as shown in (d) of FIG. 1, the entire brick / tile can constitute one slice (all:1 partitioning relationship), and as shown in (e) of FIG. 1, multiple bricks / tiles can form one raster scan slice. In (e) of FIG. 1, the bricks / tiles filled with diagonal lines in the upper right direction constitute one raster scan slice. Further, among the bricks / tiles not filled with the diagonal line pattern, the two bricks / tiles on the upper part of the picture constitute another raster scan slice, and the other 3 bricks / tiles on the lower part and the right side of the picture constitute yet another raster scan slice. Figure 12 Figure 12 Figure 12 Figure 12 Figure 12 Figure 12

[0161] The video encoding apparatus determines whether the slice includes one or more bricks / tiles, and sets the determination result as the value of the single_brick_per_slice_flag syntax element. When single_brick_per_slice_flag = = 1, it indicates that the slice contains one brick / tile, and when single_brick_per_slice_flag = = 0, it indicates that the slice can contain one or more bricks / tiles. The single_brick_per_slice_flag is encoded and signaled to the video decoding apparatus.

[0162] ​​​​​​When single_brick_per_slice_flag = = 1, the tiles / bricks and slices have a 1 : 1 partitioning relationship; therefore, the video encoding device does not signal additional slice partitioning information.

[0163] The video encoding device determines the mode of a slice and sets the determination result as the value of a rect_slice_flag syntax element. When rect_slice_flag = = 0, it indicates that the mode of the relevant slice is a raster scan slice mode; when rect_slice_flag = = 1, it indicates that the mode of the relevant slice is a rectangular slice mode. When single_brick_per_slice_flag = = 0, rect_slice_flag is encoded and signaled to the video decoding device.

[0164] The video encoding device determines the number of tiles in a picture and sets the determination result as the value of a num_slices_in_pic_minusl syntax element. num_slices_in_pic_minusl is a syntax element indicating a value obtained by subtracting 1 from the number of slices in a picture. num_slices_in_pic_minusl is encoded and signaled to the video decoding device.

[0165] The video encoding device determines the tiles / bricks located in the right lower part of each slice (right lower tiles / bricks) and sets the absolute value and the sign of the index value difference between the right lower tiles / bricks as the values of bottom_right_brick_idx_delta[i] syntax elements and brick_idx_delta_sign_flag[i] syntax elements.

[0166] For the second slice and subsequent (i>1) slices in a picture, bottom_right_brick_idx_delta[i] indicates the absolute value of the difference between the index of the bottom-right brick / tile in the previous slice and the index of the bottom-right brick / tile in the subsequent slice. However, bottom_right_brick_idx_delta[i] indicates the index of the bottom-right brick / tile in the first slice (i==0) in a picture when it reaches the first slice. When brick_idx_delta_sign_flag[i] == 1, it indicates that the sign of bottom_right_brick_idx_delta[i] is positive; when brick_idx_delta_sign_flag[i] == 0, it indicates that the sign of bottom_right_brick_idx_delta[i] is negative. Both bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] are coded and signaled to the video decoding device.

[0167] Before bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] are signaled, a bottom_right_brick_idx_length_minus1 syntax element is coded and signaled. bottom_right_brick_idx_length_minus1 indicates the number of bits used to represent bottom_right_brick_idx_delta[i].

[0168] Decoder 410 decodes single_brick_per_slice_flag (S1310) from the bitstream. When single_brick_per_slice_flag is not present in the bitstream, the value of single_brick_per_slice_flag is set or derived to be "1".

[0169] The decoder 410 determines whether to decode (obtain) additional slice partition information based on the value of single_brick_per_slice_flag (S1320). When single_brick_per_slice_flag == 1, the additional slice partition information is not decoded, and the slice partition information is derived from the tile / brick partition information. In contrast, when single_brick_per_slice_flag == 0, the decoder 410 decodes rect_slice_flag from the bitstream (S1330). When rect_slice_flag is not present in the bitstream, the value of rect_slice_flag is set or derived to be "1".

[0170] When the mode of the slice is the rectangular slice mode (rect_slice_flag == 1) and the slice includes one or more tiles / bricks (single_brick_per_slice_flag == 0) (S1340), the decoder 410 decodes num_slices_in_pic_minus1 and bottom_right_brick_idx_length_minus1 from the bitstream (S1350). When num_slices_in_pic_minus1 is not present in the bitstream and single_brick_per_slice_flag == 1, num_slices_in_pic_minus1 is not decoded, and is set or derived to be the value obtained by subtracting 1 from the number of tiles / bricks in the picture.

[0171] When the mode of the slice is the rectangular slice mode (rect_slice_flag == 1) and the slice includes one or more tiles / bricks (single_brick_per_slice_flag == 0) (S1340), the decoder 410 decodes bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] from the bitstream (S1360). Since bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] are not signaled (decoded) when single_brick_per_slice_flag == 1, bottom_right_brick_idx_delta[i] is set or derived to be 1 (bottom_right_brick_idx_delta[i] == 1).

[0172] Table 6 shows a slice partitioning method of embodiment 2-1 of the partitioning relationship between the bricks and slices of Figure 12

[0173] [Table 6]

[0174]

[0175] In the case of (b) of Figure 12 , since the partitioning relationship between the bricks / tiles and slices is the same (1:1 relationship), it is signaled by setting single_brick_per_slice_flag to “1”; the other slice partitioning information is derived from the brick / tile partitioning information in the absence of signaling.

[0176] In the case of (c) of Figure 12 , since the partitioning relationship between the bricks / tiles and slices is not the same, it is signaled by setting single_brick_per_slice_flag to “0”; since the slice mode is the rectangular slice mode, it is signaled by setting rect_slice_flag to 1. In addition, since the number of slices in the picture is 3, it is signaled by setting num_slices_in_pic_minus1 to 2; bottom_right_brick_idx_delta[i] is set and signaled such that bottom_right_brick_idx_delta[0]==2, bottom_right_brick_idx_delta[1]==3, and bottom_right_brick[2]==3. In addition, brick_idx_delta_sign_flag[i] is set and signaled such that brick_idx_delta_sign_flag[0]==1(+), brick_idx_delta_sign_flag[1]==1, and brick_idx_delta_sign_flag[2]==1.

[0177] In the cases of (d) and (e) of Figure 12 , since the partitioning relationship between the bricks / tiles and slices is not the same, it is signaled by setting single_brick_per_slice_flag to “0”; since the slice mode is the raster scan slice mode, it is signaled by setting rect_slice_flag to 0. The remaining slice partitioning information is not signaled.

[0178] In different embodiments, Figure 12 ​The slice mode of (d) can be set to a rectangular slice mode (rect_slice_flag == 1), and the number of slices in the picture can be set to 1 (num_slices_in_pic_minus1 == 0). In this case, Table 7 shows an example of signaled slice partitioning information.

[0179] [Table 7]

[0180] single_brick_per_slice_flag of (d) rect_slice_flag 0 num_slices_in_pic_minus1 1 bottom_right_brick_idx_length_minus1 0 bottom_right_brick_idx_delta 0 brick_idx_delta_sign_flag - Figure 12 -

[0181] Embodiment 2-2

[0182] Embodiment 2-2 relates to a method for obtaining a flag (loop_filter_across_slices_enabled_flag) indicating whether to apply in-loop filtering across boundaries between slices.

[0183] Whether to signal loop_filter_across_slices_enabled_flag can be determined based on a preset condition. This is because, depending on the partitioning relationship between tiles / slices and slices, the boundaries of tiles / slices and the boundaries of slices can partially or completely overlap (or can be the same). For example, in the case of (b) of Figure 12 , the boundaries of tiles / slices and the boundaries of slices are completely the same; in the case of (c) of Figure 12 , the boundaries of tiles / slices and the boundaries of slices are partially the same; in the case of (d) of Figure 12 , the boundaries of slices become the boundaries of the picture, and in the case of (e) of Figures 14 to 17 , the boundaries of tiles / slices and the boundaries of slices can be partially the same.

[0184] Various examples of Embodiment 2-2 are shown in Figure 14

[0185] The video encoding apparatus encodes slice partitioning information and signals the encoded information to the video decoding apparatus. The decoder 410 decodes the slice partitioning information from the bitstream (S1410, S1510, S1610, S1710) and partitions the picture into one or more slices based on the slice partitioning information.

[0186] ​Depending on preset conditions, the video encoding device determines whether to signal `loop_filter_across_slices_enabled_flag`. In other words, when the preset conditions are met, `loop_filter_across_slices_enabled_flag` is encoded and signaled to the video decoding device. Decoder 410 determines whether the preset conditions are met (S1420, S1520, S1620, S1720), and decodes `loop_filter_across_slices_enabled_flag` from the bitstream based on the determination result (S1430, S1530, S1630, S1730).

[0187] The preset conditions may include one or more conditions such as whether there are multiple slices in the image (whether the image is divided into multiple slices) and whether the slice mode is a rectangular slice mode.

[0188] For example, such as Figure 12 As shown, the preset conditions may include whether there are multiple slices in the frame (num_slices_in_pic_minus1) and whether the slice mode is rectangular slice mode (rect_slice_flag). In this case, when there are multiple slices in the frame or the slice mode is not rectangular slice mode (num_slices_in_pic_minus1>0||rect_slice_flag==0), a signal is sent to notify and decode loop_filter_across_slices_enabled_flag.

[0189] exist Figure 12 In cases (b) and (c), since num_slices_in_pic_minus1 > 0, the loop_filter_across_slices_enabled_flag is signaled and decoded, and... Figure 15 In cases (d) and (e), since rect_slice_flag == 0, loop_filter_across_slices_enabled_flag is signaled and decoded.

[0190] In different implementations, the preset conditions may include whether the slicing mode is a rectangular slicing mode, whether the slice includes one or more tiles / blocks, and whether one or more conditions are applied to the boundaries between tiles / blocks with in-loop filtering.

[0191] For example, such as Figure 15As shown, the preset condition can include all of the following conditions: whether the slice mode is the rectangular slice mode (rect_slice_flag), whether one or more bricks / tiles are included in the slice (single_brick_per_slice_flag), and whether the in-loop filtering is applied to the boundary between the bricks / tiles (loop_filter_across_bricks_enabled_flag).

[0192] In this case, when the slice mode is the rectangular slice mode, the slice includes one or more bricks / tiles, and the in-loop filtering is applied to the boundary between the bricks / tiles (rect_slice_flag == 1 && single_brick_per_slice_flag == 0 && loop_filter_across_bricks_enabled_flag == 1), the loop_filter_across_slices_enabled_flag is signaled and decoded. Table 5 shows the syntax structure of the preset condition as shown. Figure 12

[0193] In the case of (b) of Figure 12 Since single_brick_per_slice_flag == 1, the loop_filter_across_slices_enabled_flag is not signaled and decoded. Since the partitioning relationship between the bricks / tiles and the slice is 1:1 (i.e., all the boundaries of the bricks / tiles coincide with the boundaries of the slice), there is no need to signal the loop_filter_across_slices_enabled_flag for the slice boundary separately.

[0194] In the case of (c) of Figure 12 Since rect_slice_flag == 1, single_brick_per_slice_flag == 0, and loop_filter_across_bricks_enabled_flag == 1, the loop_filter_across_slices_enabled_flag is signaled. In the case of (d) and (e) of Figure 12 Since rect_slice_flag == 0, the loop_filter_across_slices_enabled_flag is not signaled. Here, in the case of (d) of Figure 16 ​In the case of the full: 1 split relationship shown in (d) of FIG. 1, the slice boundary coincides with the picture boundary, and thus loop_filter_across_slices_enabled_flag for the slice boundary does not need to be separately signaled.

[0195] In different embodiments, as shown in (a) to (e) of FIG. 1, the preset condition can include whether one or more bricks / slices are included in a slice (single_brick_per_slice_flag) and whether loop_filter_across_slices_enabled_flag is signaled. Figure 16

[0196] In this case, when one or more bricks / slices are included in the slice, and loop_filter_across_slices_enabled_flag is signaled (single_brick_per_slice_flag == 1 && loop_filter_across_slices_enabled_flag == 1), loop_filter_across_slices_enabled_flag is signaled and decoded.

[0197] Table 8 shows the syntax structure of the preset condition shown in (d) of FIG. 1. Figure 12

[0198] [Table 8]

[0199]

[0200] In the case of (b) of FIG. 1, since single_brick_per_slice_flag == 1, loop_filter_across_slices_enabled_flag is not signaled. In the cases of (c) to (e) of FIG. 1, since single_brick_per_slice_flag == 0 and loop_filter_across_bricks_enabled_flag == 1, loop_filter_across_slices_enabled_flag is signaled. Figure 12 Figure 17 In the case of (b) of FIG. 1, since single_brick_per_slice_flag == 1, loop_filter_across_slices_enabled_flag is not signaled. In the cases of (c) to (e) of FIG. 1, since single_brick_per_slice_flag == 0 and loop_filter_across_bricks_enabled_flag == 1, loop_filter_across_slices_enabled_flag is signaled.

[0201] In different embodiments, as shown in (a) to (e) of FIG. 1, the preset condition can include whether one or more bricks / slices are included in a slice (single_brick_per_slice_flag) and whether loop_filter_across_slices_enabled_flag is signaled. Figure 17 ​​​As shown, the preset condition can include whether the number of slices in a picture (num_slices_in_pic_minus1), whether the slice mode is a rectangular slice mode (rect_slice_flag), whether one or more bricks / squares are included in a slice (single_brick_per_slice_flag), and whether to apply in-loop filtering across the boundaries between bricks / squares (loop_filter_across_bricks_enabled_flag).

[0202] In this case, when in-loop filtering across the boundaries between bricks / squares is applied, one or more bricks / squares are included in a slice, and the slice mode is not a rectangular slice mode (loop_filter_across_bricks_enabled_flag == 1 && single_brick_per_slice_flag == 0 && rect_slice_flag == 0), loop_filter_across_slice_enabled_flag is signaled and decoded.

[0203] Alternatively, when in-loop filtering across the boundaries between bricks / squares is applied, one or more bricks / squares are included in a slice, the slice mode is a rectangular slice mode, and the number of slices in a picture is multiple (loop_filter_across_bricks_enabled_flag == 1 && single_brick_per_slice_flag == 0 && rect_slice_flag == 1 && num_slices_in_pic_minus1 > 0), loop_filter_across_slices_enabled_flag is signaled and decoded.

[0204] Table 9 shows Figure 17 the syntax structure of the preset condition.

[0205] [Table 9]

[0206]

[0207] If the slice setting shown in Table 7 is applied to the preset condition of Figure 12 , whether to signal the enabling flag can be determined as follows. In the case of (b) of Figure 12 , since single_brick_per_slice_flag == 1, loop_filter_across_slices_enabled_flag is not signaled. In the case of (c) of , since loop_filter_across_bricks_enabled_flag == 1, loop_filter_across_slices_enabled_flag is signaled.Figure 12 In cases (c) and (e), since loop_filter_across_bricks_enabled_flag == 1, single_brick_per_slice_flag == 0, rect_slice_flag == 1, and num_slices_in_pic_minus1 > 0, loop_filter_across_slices_enabled_flag is signaled. Figure 18 In case (d), since num_slices_in_pic_minus1 == 0, no signal is needed to notify loop_filter_across_slices_enabled_flag.

[0208] Implementation Method 3

[0209] Implementation method 3 relates to a method for dividing a screen into sub-screens (a method for obtaining sub-screen segmentation information). Figure 19 An example of implementation method 3 is shown.

[0210] After dividing the frame into multiple CTUs of predetermined size, the video encoding device encodes the size information of the CTUs (CTU size information) and signals this encoded information to the video decoding device. Furthermore, after dividing the CTUs into CUs based on a tree structure, the video encoding device encodes the minimum size information of the CUs (MinCb size information) and signals this encoded information to the video decoding device. The MinCb size information can be defined and signaled at one or more locations at the SPS level, PPS level, and slice header level of the bitstream.

[0211] The video encoding device divides the screen into multiple sub-screens. The sub-screens can be based on the CTU size derived from CTU size information or on the MinCb size derived from MinCb size information.

[0212] Therefore, sub-screen segmentation information can represent sub-screens within a frame in units of CTU size or MinCb size. Sub-screen segmentation information can include sub-screen position information indicating the location of each sub-screen and sub-screen size information indicating the size of each sub-screen. Sub-screen position information can represent the location of a sub-screen within a frame in units of CTU size or MinCb size. Sub-screen size information can also represent the size of a sub-screen in units of CTU size or MinCb size.

[0213] The decoder 410 decodes CTU size information and MinCb size information from the bitstream (S1810). In addition, the decoder 410 decodes subpicture partitioning information from the bitstream (S1820).

[0214] The decoder 410 partitions a picture into CTUs based on the CTU size information, and partitions the picture into subpictures based on the subpicture partitioning information (S1830).

[0215] A conventional subpicture partitioning method represents a subpicture in units of 4 samples. In other words, the conventional subpicture partitioning method uses a multiple of 4 as a unit of subpicture partitioning. However, as described above, a picture can be partitioned into a plurality of CTUs, partitioned into tiles / slices including one or more CTUs, partitioned into slices including one or more tiles / slices, and partitioned into subpictures including one or more slices. Accordingly, it can be most efficient to represent tiles / slices, slices, and subpictures in units of CTU size (in multiples of CTU). Accordingly, when a subpicture is represented in units of CTU size according to Embodiment 3, efficient subpicture partitioning can be achieved.

[0216] Based on this embodiment, subpicture partitioning information can directly or indirectly represent a subpicture in units of CTU size or MinCb size. Here, the indirect representation method connects a CTU and a subpicture through different subunits. The different subunits can be a grid, and the grid can be represented in units of CTU size, a subpicture is represented in units of grid size, and as a result, a subpicture can be represented in units of CTU size.

[0217] ​ An example of the indirect representation method is shown, and Table 10 shows a syntax structure of the indirect representation method.

[0218] [Table 10]

[0219]

[0220] A video encoding apparatus determines whether subpicture partitioning is performed (whether a subpicture parameter is present in a bitstream), and sets a result of the determination as a value of a subpics_present_flag syntax element. When subpics_present_flag == 1, it indicates that subpicture partitioning is performed, and when subpics_present_flag == 0, it indicates that subpicture partitioning is not performed.

[0221] When subpics_present_flag == 1, the video encoding device decodes max_subpics_minus1 and subpic_grid_row_height_minus1, where max_subpics_minus1 is a syntax element indicating the maximum number of sub-pictures, subpic_grid_col_width_minus1 is a syntax element indicating the width of a grid column in CTU size or in MinCb size, and subpic_grid_row_height_minus1 is a syntax element indicating the height of a grid row in CTU size or in MinCb size, and the coded syntax elements are signaled to the video decoding device.

[0222] max_subpics_minus1 represents a value obtained by subtracting 1 from the maximum number of sub-pictures, subpic_grid_col_width_minus1 represents a value obtained by subtracting 1 from the width of a grid column, and subpic_grid_row_height_minus1 represents a value obtained by subtracting 1 from the height of a grid row.

[0223] The video encoding device encodes sub-picture indexes (subpic_grid_idx[i][j]) and signals the coded sub-picture indexes to the video decoding device. The sub-picture indexes indicate the indexes of the sub-pictures to which the respective grids belong, and it can be determined which grid belongs to which sub-picture through the sub-picture indexes.

[0224] The decoder 410 decodes subpics_present_flag from the bitstream (S1910) and determines the value of subpics_present_flag (S1920).

[0225] When subpics_present_flag == 1, the decoder 410 decodes max_subpics_minus1, subpic_grid_col_width_minus1, and subpic_grid_row_height_minus1 from the bitstream (S1930).

[0226] Since the grid has a fixed width and height, the number of grid columns (i) can be derived based on subpic_grid_col_width_minus1, and the number of grid rows (j) can be derived based on subpic_grid_row_height_minus1. As a result, the picture can be partitioned into a grid based on subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1.

[0227] The decoder 410 can decode a subpicture index from the bitstream (S1940), and partition the picture into subpictures by determining, from among the plurality of grids, a grid to which the same subpicture index is assigned as a subpicture.

[0228] Although the exemplary embodiments have been described for illustrative purposes, those skilled in the art will appreciate that various modifications and changes are possible without departing from the idea and scope of the embodiments. The exemplary embodiments have been described for the sake of brevity and clarity. Those skilled in the art will appreciate that the scope of the embodiments is not limited to the embodiments explicitly described above, but includes the claims and their equivalents.

[0229] Cross Reference to Related Applications

[0230] This application claims priority to Korean Patent Applications 10-2019-0119353 and 10-2020-0113632, filed on September 27, 2019 and September 7, 2020, respectively, the entire contents of which are incorporated herein by reference.

Claims

1. A video decoding method, comprising the following steps: Decode information about the subunits of the frame from the bitstream; as well as The image is reconstructed based on the information about the sub-unit. The step of decoding the information about the sub-unit includes the following steps: Decode the CTU size information from the bitstream, indicating the size of the Code Tree Unit (CTU) within the frame. Decoding sub-picture segmentation information from the bitstream, the sub-picture segmentation information being used to segment the picture into one or more sub-pictures, each of the one or more sub-pictures consisting of at least one slice, wherein the sub-picture segmentation information includes sub-picture size information representing the size of each sub-picture in units of CTU size. Decode from the bitstream tile segmentation information relating to dividing the image into one or more tiles, wherein the tile segmentation information includes at least one of information about the number of tiles and information about the size of the tiles. Decode from the bitstream slice segmentation information about dividing the image into one or more slices, and Based on the slice pattern among multiple slice patterns including rectangular slice pattern and raster scan slice pattern, a bitstream decoding enable flag is generated, which indicates whether filtering is performed at the boundary of the slice.

2. The video decoding method according to claim 1, wherein, The slice segmentation information is defined based on the pieces determined by the piece segmentation information.

3. The video decoding method according to claim 2, wherein, When there are multiple slices and the slice mode is the rectangular slice mode, or when the slice mode is the raster scan slice mode, the enable flag is decoded.

4. A video encoding method, comprising the following steps: The image is divided into sub-units for encoding; as well as Information about the subunit is encoded; The step of encoding the information about the sub-unit includes: The CTU size information, which indicates the size of the coding tree unit (CTU) within the image, is encoded. The sub-screen segmentation information is encoded to divide the screen into one or more sub-screens, each of which consists of at least one slice. The sub-screen segmentation information includes sub-screen size information, expressed in CTU size units, representing the size of each sub-screen. The tile segmentation information relating to dividing the image into one or more tiles is encoded, wherein the tile segmentation information includes at least one of information about the number of tiles and information about the size of the tiles. Encode the slice segmentation information that divides the image into one or more slices, and Based on the slice pattern among multiple slice patterns, including rectangular slice pattern and raster scan slice pattern, an enable flag is encoded, which indicates whether filtering is performed at the boundary of the slice.

5. A method for transmitting a bitstream comprising encoded video data, the method comprising the steps of: The bitstream is generated by dividing the image into sub-units for encoding and encoding information about the sub-units; as well as The bitstream is sent to the video decoding device. The step of encoding the information about the sub-unit includes the following steps: The CTU size information, which indicates the size of the coding tree unit (CTU) within the image, is encoded. The sub-screen segmentation information is encoded to divide the screen into one or more sub-screens, each of which consists of at least one slice. The sub-screen segmentation information includes sub-screen size information, expressed in CTU size units, representing the size of each sub-screen. The tile segmentation information relating to dividing the image into one or more tiles is encoded, wherein the tile segmentation information includes at least one of information about the number of tiles and information about the size of the tiles. Encode the slice segmentation information that divides the image into one or more slices, and Based on the slice pattern among multiple slice patterns, including rectangular slice pattern and raster scan slice pattern, an enable flag is encoded, which indicates whether filtering is performed at the boundary of the slice.

Images