Video decoding / encoding method and bitstream transmission device

Abstract

The invention provides a video decoding / encoding method and a bitstream transmission device. A method and an apparatus for acquiring information on subunits split from a screen are disclosed. According to an embodiment of the present invention, a method for acquiring information on subunits split from a screen is provided. The method comprises the steps of: decoding, from a bitstream, CTU size information indicating a size of a coding tree unit (CTU) within a picture; decoding, from a bitstream, sub-picture splitting information indicating sub-pictures within the picture in units of the CTU size; decoding splitting information of one or more tiles within the picture from a bitstream; and decoding splitting information of one or more slices within the picture from a bitstream.

Description

[0001] This application is a divisional application of the original patent application No. 202080082074.8 (International Application No.: PCT / KR2020 / 012769, Application Date: September 22, 2020, Invention Title: Method and Apparatus for Obtaining Information about Sub-units Split from a Screen). Technical Field

[0002] This disclosure relates to video encoding and decoding, and more specifically, to methods and apparatus for improving encoding and decoding efficiency by efficiently obtaining information about various subunits. Background Technology

[0003] Because video data is larger than audio or still image data, storing or transmitting video data without compression requires significant hardware resources, including memory.

[0004] Therefore, when storing or transmitting video data, an encoder is typically used to compress the video data for storage or transmission. The decoder then receives the compressed video data and decompresses and reproduces it. Technologies used for this video compression include H.264 / AVC and High Efficiency Video Coding (HEVC), which improves coding efficiency by approximately 40% compared to H.264 / AVC.

[0005] However, video size, resolution, and frame rate are gradually increasing, and consequently, the amount of data to be encoded is also increasing. Therefore, there is a need for a new compression technique that offers better encoding efficiency and higher image quality than existing compression techniques. Summary of the Invention

[0006] Technical issues

[0007] The purpose of this disclosure is to provide an improved video encoding and decoding technique to meet this need. Specifically, one aspect of this disclosure relates to techniques for improving encoding and decoding efficiency by various methods of dividing a frame into sub-units.

[0008] Technical solution

[0009] According to one aspect, this disclosure provides a method for obtaining information about sub-units segmented from a frame, the method comprising: decoding from a bitstream CTU size information indicating the size of a coding tree unit (CTU) within the frame; decoding from the bitstream sub-frame segmentation information representing sub-frames within the frame in units of the CTU size; decoding from the bitstream segmentation information about dividing the frame into one or more tiles; and decoding from the bitstream segmentation information about dividing the frame into one or more slices.

[0010] According to another aspect, the present invention provides a decoder configured to decode from a bitstream information about sub-units segmented from a frame; and a frame reconstructor configured to reconstruct the frame based on information about the sub-units, wherein the decoder is configured to decode from the bitstream CTU size information indicating the size of a coding tree unit (CTU) within the frame, decode from the bitstream sub-frame segmentation information representing sub-frames within the frame in units of CTU size, decode from the bitstream segmentation information about segmenting the frame into one or more tiles; and decode from the bitstream segmentation information about segmenting the frame into one or more slices.

[0011] Technical effect

[0012] As described above, according to one embodiment of this disclosure, bit efficiency can be improved by preventing the overlap of syntactic elements used to divide the screen into subunits.

[0013] Furthermore, according to another embodiment of this disclosure, whether to apply in-loop filtering between sub-unit boundaries can be reasonably designed based on the segmentation relationship between sub-units.

[0014] Furthermore, according to another embodiment of this disclosure, since the segmentation information of the sub-screen is represented in units of CTU, consistency with other sub-units can be ensured, and the segmentation information of the sub-screen can be represented more efficiently through the other sub-units. Attached Figure Description

[0015] Figure 1 This is a schematic block diagram of a video encoding device capable of implementing the technology of the present invention.

[0016] Figure 2 An exemplary block partitioning structure using the QTBTTT structure is shown.

[0017] Figure 3 Several intra-frame prediction modes are illustrated as examples.

[0018] Figure 4 This is a schematic block diagram of a video decoding device capable of implementing the technology of the present invention.

[0019] Figures 5 to 8 It shows slices, tiles, and sub-pictures.

[0020] Figure 9 This is a flowchart illustrating an example of a method capable of implementing the techniques disclosed herein.

[0021] Figure 10 This is a flowchart illustrating an example of a method for dividing a screen into tiles.

[0022] Figure 11 This is a flowchart illustrating an example of a method for dividing a screen into blocks.

[0023] Figure 12 This illustrates the division relationship between puzzle pieces / blocks and slices.

[0024] Figure 13 This is a flowchart illustrating an example of a method for dividing a screen into slices.

[0025] Figures 14 to 17 This is a flowchart illustrating various examples of methods for obtaining information about whether a filter is applied to slice boundaries.

[0026] Figure 18 This is a flowchart illustrating an example of a method for dividing a screen into sub-screens.

[0027] Figure 19 This is a flowchart illustrating a specific method for dividing a screen into sub-screens. Detailed Implementation

[0028] In the following description, some embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that when reference numerals are added to the constituent elements in the various drawings, the same reference numerals denote the same elements, even though these elements are shown in different drawings. Furthermore, in the following description of the present disclosure, detailed descriptions of known functions and configurations incorporated herein will be omitted to avoid obscuring the subject matter of the disclosure.

[0029] Figure 1 This is a schematic block diagram of a video encoding apparatus capable of implementing the technology of the present invention. In the following text, reference will be made to... Figure 1 Describe the video encoding apparatus and its components.

[0030] The video encoding apparatus includes a block splitter 110, a predictor 120, a subtractor 130, a converter 140, a quantizer 145, an encoder 150, an inverse quantizer 160, an inverse converter 165, an adder 170, a filter unit 180, and a memory 190.

[0031] The various components of a video encoding device can be implemented using hardware, software, or a combination of both. The functions of each component can be implemented in software, and a microprocessor can be implemented to execute the software functions corresponding to each component.

[0032] A video consists of multiple frames. Each frame is divided into multiple regions, and encoding is performed on each region. For example, a frame is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more coding tree units (CTUs). Each CTU is further divided into one or more coding units (CUs) through a tree structure. Information applied to each CU is encoded as the syntax of the CU, while information applied to multiple CUs included in a common CTU is encoded as the syntax of the CTU. Furthermore, information applied to all blocks in a common tile is encoded as the syntax of the tile or as the syntax of the tile group, where a tile group is a collection of multiple tiles, and information applied to all blocks constituting a frame is encoded in the frame parameter set (PPS) or frame header. In addition, information commonly referenced by multiple frames is encoded in the sequence parameter set (SPS). Furthermore, information referenced by one or more common SPSs is encoded in the video parameter set (VPS).

[0033] Block splitter 110 determines the size of the coding tree unit (CTU). Information about the CTU size (CTU size) is encoded into SPS or PPS syntax and sent to the video decoding device.

[0034] The block splitter 110 splits the individual frames that make up the video into multiple CTUs of a predetermined size, and then recursively splits the CTUs using a tree structure. In the tree structure, leaf nodes are used as coding units (CUs), which are the basic units of encoding.

[0035] The tree structure can be a quadtree (QT), where a node (or parent node) is split into four sub-nodes (or child nodes) of the same size; a binary tree (BT), where a node is split into two child nodes; a ternary tree (TT), where a node is split into three child nodes in a 1:2:1 ratio; or a structure formed by a combination of two or more of the QT, BT, and TT structures. For example, a QTBT (quadtree plus binary tree) structure or a QTBTTT (quadtree plus binary tree plus ternary tree) structure can be used. In this paper, BTTT can be collectively referred to as Multi-Type Tree (MTT).

[0036] Figure 2 The QTBTTT split tree structure is shown. (Example) Figure 2As shown, the CTU can be initially split within the QT structure. QT splitting can be repeated until the size of the split blocks reaches the minimum block size (minqtsize) allowed for leaf nodes in the QT. The encoder 150 encodes a first flag (QT_split_flag) indicating whether each node in the QT structure is split into the next four nodes and signals this flag to the video decoding device. When a leaf node of the QT is not larger than the maximum block size (maxbtsize) allowed for the root node in the BT, it can be further split into one or more BT or TT structures. In the BT and / or TT structures, there can be multiple splitting directions. For example, there can be two directions: horizontal splitting and vertical splitting of node blocks. Figure 2 As shown, when MTT splitting begins, encoder 150 encodes a second flag (mtt_split_flag) indicating whether a node is split, a flag indicating the splitting direction (vertical or horizontal), and / or a flag indicating the splitting type (binary or ternary) and signals it to the video decoding device.

[0037] As another example of a tree structure, when a block is split using the QTBTTT structure, the encoder 150 encodes information about a CU split flag (split_cu_flag) indicating that the block has been split and a QT split flag (split_qt_flag) indicating whether the split type is QT splitting, and signals this information to the video decoding device. When the value of split_cu_flag indicates that the block has not been split, the block becomes a leaf node in the split tree structure and is used as a coding unit (CU), which is the basic unit of encoding. When the value of split_cu_flag indicates that the block has been split, the split type (QT or MTT) is distinguished by the value of split_qt_flag. When the split type is QT, no additional information is provided. When the split type is MTT, the flag indicating the MTT split direction (vertical or horizontal) (mtt_split_cu_vertical_flag) and / or the flag indicating the MTT split type (binary or ternary) (mtt_split_cu_binary_flag) are encoded by the encoder 150 and signaled to the video decoding device.

[0038] As another example of a tree structure, when using QTBT, there are two types of splitting: horizontally splitting a node's block (i.e., symmetrical horizontal splitting) and vertically splitting it (i.e., symmetrical vertical splitting) into two blocks of the same size. Encoder 150 encodes and sends to the video decoding device a splitting flag (split_flag) indicating whether each node in the BT structure is split into blocks of the lower level and splitting type information indicating the splitting type. There can also be additional types that split a node's block into two asymmetric blocks. Asymmetric splitting types can include types that split a block into two rectangular blocks with a 1:3 size ratio, and types that split a node's block diagonally.

[0039] CUs can have various sizes depending on the QTBT or QTBTTT split of the CTU. In the following text, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block".

[0040] Predictor 120 predicts the current block to generate a prediction block. Predictor 120 includes an intra-frame predictor 122 and an inter-frame predictor 124.

[0041] Typically, predictive coding can be performed on each current block in a frame. Prediction of the current block can be performed using either intra-frame prediction (which is based on data from the frame containing the current block) or inter-frame prediction (which is based on data from the frame encoded before the frame containing the current block). Inter-frame prediction includes one-way prediction and two-way prediction.

[0042] Intra-predictor 122 uses pixels (reference pixels) located around the current block in the current frame, including the current block, to predict pixels in the current block. Multiple intra-prediction modes exist depending on the prediction direction. For example, such as... Figure 3 As shown, multiple intra-frame prediction modes can include non-directional modes and 65 directional modes. The non-directional modes include planar modes and DC modes. For each prediction mode, neighboring pixels and the formula to be used are defined differently.

[0043] Intra-predictor 122 can determine the intra-prediction mode to be used for encoding the current block. In some examples, intra-predictor 122 can use multiple intra-prediction modes to encode the current block and select the appropriate intra-prediction mode from the tested modes. For example, intra-predictor 122 can use rate distortion analysis of multiple tested intra-prediction modes to calculate rate distortion values ​​and can select the intra-prediction mode with the best rate distortion characteristics from the tested modes.

[0044] Intra-predictor 122 selects one intra-prediction mode from a plurality of intra-prediction modes and uses neighboring pixels (reference pixels) and a formula determined according to the selected intra-prediction mode to predict the current block. Information about the selected intra-prediction mode is encoded by encoder 150 and sent to video decoding device.

[0045] Inter-frame predictor 124 generates a predicted block for the current block through motion compensation processing. The inter-frame predictor searches for the most similar block in a reference frame encoded and decoded earlier than the current frame, and generates a predicted block for the current block based on the searched block. Then, the inter-frame predictor generates a motion vector corresponding to the displacement between the current block in the current frame and the predicted block in the reference frame. Typically, motion estimation is performed on the luminance component, and the motion vector calculated based on the luminance component is used for both the luminance and chrominance components. Encoder 150 encodes motion information including information about the reference frame used to predict the current block and information about the motion vector, and sends it to the video decoding device.

[0046] Subtractor 130 generates a residual block by subtracting the prediction block generated by intra-predictor 122 or inter-predictor 124 from the current block.

[0047] Transformer 140 transforms the residual signal in the residual block with pixel values ​​in the spatial domain into transform coefficients in the frequency domain. Transformer 140 can use the total size of the current block as a transform unit to transform the residual signal in the residual block. Alternatively, the transformer can split the residual block into sub-blocks of transformed and non-transformed regions, and use only the sub-blocks of the transformed regions as transform units to transform the residual signal. Here, the sub-block of the transformed region can be one of two rectangular blocks with a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, encoder 150 encodes a flag (cu_sbt_flag) indicating that only the sub-block has been transformed, orientation (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) and signals it to the video decoding device. Furthermore, the size of the sub-block of the transformed region can have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, the encoder 150 additionally encodes the flag (cu_sbt_quad_flag) used for distinguishing the split and sends a signal to the video decoding device.

[0048] The quantizer 145 quantizes the transformation coefficients output from the converter 140 and outputs the quantized transformation coefficients to the encoder 150.

[0049] Encoder 150 generates a bitstream by encoding the quantized transform coefficients using an encoding method such as context-based adaptive binary arithmetic coding (CABAC). Encoder 150 encodes information related to block splitting, such as CTU size, CU splitting flag, QT splitting flag, MTT splitting direction, and MTT splitting type, so that the video decoding device splits the blocks in the same way as the video encoding device.

[0050] In addition, encoder 150 encodes information about the prediction type that indicates whether the current block is encoded by intra-frame prediction or inter-frame prediction, and encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (information about the reference frame and motion vectors) according to the prediction type.

[0051] Inverse quantizer 160 inverse quantizes the quantized transform coefficients output from quantizer 145 to generate transform coefficients. Inverse transformer 165 transforms the transform coefficients output from inverse quantizer 160 from the frequency domain to the spatial domain and reconstructs the residual block.

[0052] Adder 170 adds the reconstructed residual block to the prediction block generated by predictor 120 to reconstruct the current block. Pixels in the reconstructed current block are used as reference pixels for intra-frame prediction of the next block.

[0053] Filter unit 180 filters the reconstructed pixels to reduce block artifacts, ringing artifacts, and blur artifacts caused by block-based prediction and transform / quantization. Filter unit 180 may include a deblocking filter 182 and a sample adaptive offset (SAO) filter 184.

[0054] Deblocking filter 182 filters the boundaries between reconstructed blocks to remove block artifacts caused by block-by-block encoding / decoding, and SAO filter 184 further filters the deblocked video. SAO filter 184 is a filter used to compensate for the differences between reconstructed pixels and original pixels caused by lossy encoding.

[0055] The reconstructed blocks filtered by deblocking filter 182 and SAO filter 184 are stored in memory 190. When all blocks in a frame are reconstructed, the reconstructed frame is used as a reference frame for inter-frame prediction of blocks in the next frame to be encoded.

[0056] Simultaneously, the image encoder divides the image into sub-units and encodes the image in units of CUs included within the sub-units. The image encoder may include all components of the video encoding apparatus other than encoder 150. Encoder 150 encodes information about the sub-units divided from the image and signals the encoded information to the video decoding apparatus. The method by which encoder 150 encodes information about the sub-units will be described in detail below.

[0057] Figure 4 This is a schematic functional block diagram of a video decoding apparatus capable of implementing the technology of the present invention. In the following text, reference will be made to... Figure 4 Describe the video decoding device and its components.

[0058] The video decoding device may include a decoder 410, an inverse quantizer 420, an inverse converter 430, a predictor 440, an adder 450, a filter unit 460, and a memory 470.

[0059] and Figure 1 Similar to video encoding devices, the components of a video decoding device can be implemented as hardware or software, or a combination of both. Furthermore, the functions of each component can be implemented as software, and a microprocessor can be implemented to execute the software functions corresponding to each component.

[0060] Decoder 410 determines the current block to be decoded by decoding the bitstream received from the video encoding device and extracting information related to block splitting, and extracts the prediction information required to reconstruct the current block and information about the residual signal.

[0061] Decoder 410 extracts information about the CTU size from the Sequence Parameter Set (SPS) or Picture Parameter Set (PPS), determines the size of the CTU, and splits the picture into CTUs of the determined size. Then, the decoder identifies the CTU as the top level, i.e., the root node of the tree structure, and extracts information about the CTU splitting to split the CTU using the tree structure.

[0062] For example, when using the QTBTTT structure to split the CTU, firstly, the first flag (QT_split_flag) related to QT splitting is extracted, and each node is split into four lower-level nodes. Then, for the node corresponding to the leaf node of QT, the second flag (MTT_split_flag) related to MTT splitting and information about the splitting direction (vertical / horizontal) and / or splitting type (binary / ternary) are extracted, and the leaf node is split in the MTT structure. In this way, the nodes below the leaf nodes of QT are recursively split using either a BT structure or a TT structure.

[0063] As another example, when splitting a CTU using a QTBTTT structure, the CU split flag (split_cu_flag) indicating whether a CU has been split is first extracted. If the corresponding block is split, the QT split flag (split_qt_flag) is extracted. When the split type is not QT but MTT, a flag indicating the MTT split direction (vertical or horizontal) (mtt_split_cu_vertical_flag) and / or a flag indicating the MTT split type (binary or ternary) (mtt_split_cu_binary_flag) are also extracted. During the splitting process, each node can undergo zero or more recursive QT splits, followed by zero or more recursive MTT splits. For example, a CTU can be an immediate MTT split, or it can simply be multiple QT splits.

[0064] As another example, when splitting a CTU using a QTBT structure, the first flag (QT_split_flag) is related to the QT split, and each node is split into four nodes at the lower level. For a node corresponding to a leaf node of the QT, the split_flag indicating whether the node is to be further split by BT and the split direction information are extracted.

[0065] When determining the current block to be decoded via tree structure splitting, decoder 410 extracts information about the prediction type, indicating whether the current block is predicted intra-frame or inter-frame. When the prediction type information indicates intra-frame prediction, decoder 410 extracts the syntax elements for the intra-frame prediction information (intra-frame prediction mode) for the current block. When the prediction type information indicates inter-frame prediction, decoder 410 extracts the syntax elements for the inter-frame prediction information, i.e., information indicating the motion vector and the reference frame referenced by the motion vector.

[0066] Decoder 410 extracts information about the quantization transform coefficients of the current block as information about the residual signal.

[0067] The inverse quantizer 420 inverse quantizes the quantized transform coefficients and inverse transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to reconstruct the residual signal, thereby generating a residual block for the current block.

[0068] Furthermore, when the inverse transformer 430 performs inverse transformation only on a portion of the transform block (sub-block), it extracts a flag indicating that only sub-blocks of the transform block have been transformed (cu_sbt_flag), as well as orientation information (vertical / horizontal) (cu_sbt_horizontal_flag) and / or sub-block position information (cu_sbt_pos_flag) about the sub-block. Then, the residual signal is reconstructed by inversely transforming the transform coefficients of the sub-blocks from the frequency domain to the spatial domain. For regions that have not been inversely transformed, the residual signal is padded with "0". Thus, the final residual block for the current block is created.

[0069] Predictor 440 may include intra-predictor 442 and inter-predictor 444. Intra-predictor 442 is activated when the prediction type of the current block is intra-prediction, and inter-predictor 444 is activated when the prediction type of the current block is inter-prediction.

[0070] Intra-predictor 442 determines the intra-prediction mode of the current block among multiple intra-prediction modes based on the syntactic elements for the intra-prediction mode extracted from decoder 410, and predicts the current block based on reference pixels around the current block according to the intra-prediction mode.

[0071] The inter-frame predictor 444 determines the motion vector of the current block and the reference frame referenced by the motion vector based on the syntactic elements of the inter-frame prediction information extracted from the decoder 410, and predicts the current block based on the motion vector and the reference frame.

[0072] Adder 450 reconstructs the current block by adding the residual block output from the inverse transform to the prediction block output from the inter-frame predictor or intra-frame predictor. Pixels in the reconstructed current block are used as reference pixels for intra-frame prediction of blocks to be decoded later.

[0073] Filter unit 460 may include a deblocking filter 462 and a SAO filter 464. Deblocking filter 462 performs deblocking filtering on the boundaries between reconstructed blocks to remove block artifacts caused by block-by-block decoding. SAO filter 464 performs additional filtering on the reconstructed blocks after deblocking filtering to compensate for differences between reconstructed pixels and original pixels caused by lossy encoding. The reconstructed blocks filtered by deblocking filter 462 and SAO filter 464 are stored in memory 470. When all blocks in a frame are reconstructed, the reconstructed frame is used as a reference frame for inter-frame prediction of blocks in subsequent frames to be encoded.

[0074] Simultaneously, decoder 410 obtains information about the sub-units segmented from the frame by decoding the bitstream. Frame reconstructor 4000 reconstructs the frame based on the obtained information about the sub-units. Frame reconstructor 4000 may include the other components in the video decoding apparatus besides decoder 410. The method by which decoder 410 obtains information about the sub-units will be described in detail below.

[0075] In the HEVC international standard, for the purpose of parallel processing, there are two types of methods for segmenting the image (or screen): slice segmentation and tile segmentation.

[0076] The slice segmentation method provides the function of dividing the image into CTUs and encoding / decoding the CTUs according to the raster scan order. Multiple slice segments can exist in a single slice segment, and there are two types of slice segments. The types of slice segments include independent slice segments and dependent slice segments.

[0077] Independent slices are slices that do not depend on another slice during intra-frame prediction, inter-frame prediction, coding mode, and entropy coding. Independent slices have header information. Dependent slices are slices that depend on another slice (referencing another previous independent slice) during intra-frame prediction, inter-frame prediction, coding mode, and entropy coding. Dependent slices reference the header information of the preceding independent slice for most of their header information. In other words, only a portion of the header information of the dependent slice is signaled.

[0078] Figure 5 (a) shows two slices divided from the image, and these two slices can be separated by slice boundaries. A slice may include one independent slice segment and zero or more dependent slice segments. The first slice at the top of the image has one independent slice segment and two dependent slice segments separated by a dashed line. The second slice at the bottom of the image has only one independent slice segment (including zero dependent slice segments).

[0079] Tables 1 and 2 show the syntactic structures used for the slice segmentation method.

[0080] [Table 1]

[0081]

[0082] [Table 2]

[0083]

[0084] Table 1 shows the flag indicating whether dependent slice segments are enabled (dependent_slice_segments_enabled_flag). This flag can be defined by PPS. Table 2 shows the slice header information.

[0085] As shown in Table 2, a flag indicating whether a specific slice is the first slice in the image (first_slice_segment_in_pic_flag) is encoded and signaled. The first slice becomes an independent slice. For slices other than the first slice, a flag indicating whether the slice is a dependent slice (dependent_slice_segment_flag) is encoded and signaled. Furthermore, the address of the related slice (slice_segment_address) is also signaled. Here, slice_segment_address represents the index value of the first CTU in the related slice.

[0086] 2) The slicing method provides encoding / decoding capabilities by dividing the image into multiple columns and rows based on CTU. The image can be segmented into patches based on a uniform segmentation scheme or a separate (non-uniform) segmentation scheme.

[0087] Each segmented piece does not depend on other pieces during intra-frame prediction and entropy coding, and its dependence on in-loop filtering can be controlled by flags defined in the PPS. There is no separate header for each piece.

[0088] Figure 5 (b) shows the screen divided into 3×3 tiles, with each tile separated by tile boundaries (column and row boundaries). Multiple tiles can be encoded / decoded in raster scan order, and multiple CTUs within a single tile can also be encoded / decoded in raster scan order. The numbers shown in each CTU indicate the order in which encoding / decoding is performed.

[0089] Table 3 shows the syntactic structures used for the chunk segmentation method.

[0090] [Table 3]

[0091]

[0092] As shown in Table 3, a flag (tiles_enabled_flag) indicating the on / off state of tile splitting is defined in PPS and signaled. When the relevant flag is on, the syntax element specifying the tile size is also signaled.

[0093] Syntactic elements that specify tile size may include num_tile_columns_minus1, num_tile_rows_minus1, and uniform_spacing_flag. num_tile_columns_minus1 indicates the value obtained by subtracting 1 from the number of tiles divided in the vertical direction (the number of tile columns), num_tile_rows_minus1 indicates the value obtained by subtracting 1 from the number of tiles divided in the horizontal direction (the number of tile rows), and uniform_spacing_flag indicates whether the screen is divided uniformly in both the horizontal and vertical directions.

[0094] When the screen is not uniformly segmented (uniform_spacing_flag = off), the width information of the tile column (column_width_minus1) and the height information of the tile row (row_height_minus1) are separately notified by a signal. In addition, a flag indicating whether in-loop filtering is applied at the boundaries between tiles (loop_filter_across_tiles_enabled_flag) is not notified by a signal.

[0095] Figure 6 The differences between the above-described slice segmentation method and the piece segmentation method are shown. Figure 6 (a) shows the slices (a total of 4 slices) segmented according to the slice segmentation method, and Figure 6 (b) shows the tiles divided according to the tile splitting method (4 tiles in total). The numbers represent the number of CTUs.

[0096] Only horizontal boundaries can exist between split slices. Conversely, horizontal and vertical boundaries can exist between split tiles.

[0097] This disclosure proposes a method for obtaining information about sub-units segmented from a picture.

[0098] A sub-unit (sub-block or sub-region) can include one or more sub-screens, slices, tiles / blocks, and CTUs. Information about a sub-unit can include its enable flag and segmentation information.

[0099] The enable flag is a syntactic element that indicates whether in-loop filtering is performed at the boundaries between sub-units. The sub-unit segmentation information corresponds to the information used to segment the screen into various sub-units. Therefore, "acquiring sub-unit segmentation information" can be "segmenting the screen into sub-units".

[0100] This disclosure proposes a screen segmentation method based on sub-unit segmentation information for various purposes. Specifically, this disclosure proposes 1) a sub-screen segmentation method for synthesizing different sequences, 2) a slice segmentation method for bitstream packets, and 3) a block / square segmentation method for parallel processing.

[0101] Tiles / blocks, slices and sub-images

[0102] A tile / block refers to a frame region obtained by dividing the frame into sub-regions for parallel processing. Tiles / blocks are processed independently during intra-frame prediction and entropy coding. In other words, tiles / blocks do not share reference samples during intra-frame prediction and do not share context during entropy coding. Whether to apply intra-loop filtering to the boundaries between tiles / blocks can be controlled by a separate flag.

[0103] A slice represents a data packet unit that configures an independent Network Abstraction Layer (NAL) bitstream. Multiple pieces / blocks can be included in a single slice, or multiple slices can be included in a single piece / block.

[0104] Slices can have two types or modes. The first mode is the rectangular slice mode, in which tiles / squares with similar characteristics or arbitrary tiles / squares are designated as a region, while the second mode is the non-rectangular slice (raster scan slice) mode, in which tiles / squares are designated as a region at a specific location, and the encoding / decoding of tiles / squares is performed in raster scan order.

[0105] Figure 7 Examples of rectangular slice mode and raster scan slice mode are shown. Figure 7 In the diagram, areas separated by dashed lines represent CTUs, areas separated by straight lines represent tiles / blocks, and areas separated by dotted lines represent slices.

[0106] like Figure 7 As shown in (a), a frame can be divided into a total of 216 (18×12) CTUs, a total of 24 (6×4) tiles / blocks, and a total of 9 rectangular slices. Figure 7 As shown in (b), a frame can be divided into a total of 216 (18×12) CTUs, a total of 12 (3×4) tiles / blocks, and a total of 3 raster scan slices.

[0107] A sub-picture is an area designated for composition with other content. A sub-picture may include at least one slice.

[0108] Figure 8This example illustrates how different content is composited using sub-pictures as units. The area filled with a diagonal line running from the upper right represents the picture in content 1. The area filled with a horizontal line pattern represents the picture in content 2. The area filled with a vertical line pattern represents the picture in content 3.

[0109] like Figure 8 As shown on the left, each frame can consist of four slices, and a bitstream can be generated for each slice unit. Furthermore, each slice can be defined as a sub-frame. The bitstream physically generated in units of slices is input to a bitstream extractor and merger, and undergoes extraction and compositing processes. Here, the logical unit of extraction and compositing is the sub-frame. The result is as follows... Figure 8 As shown on the right, a new image is generated from the composite, with the upper left part being content 1, the lower left part being content 2, and the right part being content 3.

[0110] The bitstream extractor and merger extract bitstreams generated in slices from different content and synthesize the extracted bitstreams in sub-pictures. For this purpose, the individual content frames are divided into sub-pictures. In other words, a sub-picture is a region that includes at least one slice used for content synthesis.

[0111] The method for obtaining information about sub-units will be described below. Figure 9 A flowchart illustrating one embodiment of this disclosure is shown.

[0112] Video encoding devices can divide a video frame into various sub-units. After being divided into multiple CTUs of a predetermined size, the frame can be further divided into sub-frames, tiles / blocks, and slices by using the CTU size as the basic unit.

[0113] For example, the screen can be divided into one or more sub-screens based on the CTU size as the basic unit, and the screen can also be divided into one or more tiles / squares based on the CTU size as the basic unit. In the case of slice segmentation, the screen can be divided into one or more slices based on the tiles / squares as the basic unit. Here, since the tiles / squares are segmented based on the CTU size, the slices are also segmented based on the CTU size as the basic unit.

[0114] Therefore, information about sub-screen segmentation (sub-screen segmentation information) can be represented in units of CTU size for sub-screens within the screen, information about tile / block segmentation (tile / block segmentation information) can be represented in units of CTU size for tiles / blocks within the screen, and information about slice segmentation (slice segmentation information) can be represented in units of tiles / blocks (CTU size units) for slices within the screen.

[0115] Information regarding the CTU size (CTU size information), sub-picture segmentation information, tile / block segmentation information, and slice segmentation information is encoded and signaled to the video decoding device. CTU size information is defined and signaled at the SPS or PPS level; sub-picture segmentation information is defined and signaled at the SPS level; and tile / block segmentation information and slice segmentation information are defined and signaled at the PPS level.

[0116] In addition, the video encoding device determines whether to apply in-loop filtering to the boundaries between sub-units, encodes an enable flag indicating whether to apply in-loop filtering, and signals the encoded enable flag to the video decoding device.

[0117] Define and signal an enable flag for applying in-loop filtering to the boundaries between sub-pictures at the SPS level; define and signal an enable flag for applying in-loop filtering to the boundaries between tiles / blocks and to the boundaries between slices at the PPS level.

[0118] Decoder 410 decodes the segmentation information of the subunits from the bitstream (S910). By decoding from the bitstream, CTU size information, sub-picture segmentation information, tile / block segmentation information, and slice segmentation information included in the subunit segmentation information are obtained. Decoder 410 segments the picture into subunits based on the obtained segmentation information (S930). Specifically, after being segmented into CTUs with CTU sizes indicated by the CTU size information, the picture can be segmented into sub-pictures, tiles / blocks, and slices based on the sub-picture segmentation information, tile / block segmentation information, and slice segmentation information.

[0119] Decoder 410 decodes an enable flag from the bitstream indicating whether to apply in-loop filtering to the boundaries between sub-units (S920). Filter unit 460 determines whether to apply in-loop filtering to the boundaries between sub-units based on this enable flag (S930).

[0120] Implementation Method 1

[0121] Implementation 1 discloses a method for obtaining tile segmentation information and a method for obtaining brick segmentation information. In other words, Implementation 1 relates to a method for segmenting an image into tiles and a method for segmenting an image into blocks.

[0122] Table 4 shows an example of a syntactic structure related to implementation method 1.

[0123] [Table 4]

[0124]

[0125] Implementation Method 1-1

[0126] Implementation method 1-1 relates to a method for dividing a screen into tiles (a method for obtaining tile segmentation information). Figure 10 An example of implementation method 1-1 is shown.

[0127] The video encoding device determines whether there is only one tile in the frame and sets the determination result as the value of the single_tile_in_pic_flag syntax element. When single_tile_in_pic_flag == 1, it indicates that there is only one tile in the frame, and when single_tile_in_pic_flag == 0, it indicates that there is one or more tiles in the frame.

[0128] When there are one or more tiles in the frame (single_tile_in_pic_flag == 0), the video encoding device determines whether the corresponding one or more tiles are uniformly divided and sets the determination result as the value of the uniform_tile_spacing_flag syntax element. When uniform_tile_spacing_flag == 1, it indicates that the tiles are uniformly divided from the frame, and when uniform_tile_spacing_flag == 0, it indicates that the tiles are not uniformly divided from the frame.

[0129] When tiles are uniformly segmented (uniform_tile_spacing_flag == 1), the video encoding device determines the width of the tile column and the height of the tile row, setting the results as the values ​​of the `tile_cols_width_minus1` and `tile_rows_height_minus1` syntax elements, respectively. `tile_cols_width_minus1` is a syntax element carrying the width of the tile column in CTU units, excluding the width of the rightmost tile column in the frame. `tile_rows_height_minus1` is a syntax element carrying the height of the tile row in CTU units, excluding the height of the tile row at the bottom of the frame.

[0130] When tiles are not uniformly segmented (uniform_tile_spacing_flag == 0), the video encoder determines the number of tile columns and the number of tile rows, and sets the results as the values ​​of the syntax elements num_tile_columns_minus1 and num_tile_rows_minus1, respectively. num_tile_columns_minus1 is a syntax element carrying the value obtained by subtracting 1 from the number of tile columns, and num_tile_rows_minus1 is a syntax element carrying the value obtained by subtracting 1 from the number of tile rows.

[0131] The video encoding device determines the width of the unevenly divided tile column and the height of the unevenly divided tile row, and sets the determination results as the values ​​of the `tile_column_width_minus1[i]` and `tile_row_height_minus1[i]` syntax elements, respectively. `tile_column_width_minus1[i]` is a syntax element that indicates the width of the unevenly divided tile column in CTU units. This syntax element is not defined for the width of the rightmost tile column in the frame. `tile_row_height_minus1[i]` is a syntax element that indicates the height of the unevenly divided tile row. This syntax element is not defined for the height of the tile row at the bottom of the frame.

[0132] The video encoding device encodes single_tile_in_pic_flag, uniform_tile_spacing_flag, tile_cols_width_minus1, tile_rows_height_minus1, num_tile_columns_minus1, num_tile_rows_minus1, tile_column_width_minus1[i], and tile_row_height_minus1[i], and signals these encodings to the video decoding device.

[0133] Decoder 410 decodes single_tile_in_pic_flag from the bitstream (S1010) and determines whether there is only one tile in the frame or one or more tiles in the frame based on the value of single_tile_in_pic_flag (S1020).

[0134] When there is only one tile in the frame (single_tile_in_pic_flag == 1), the decoder 410 treats the frame as a single tile and terminates the tile segmentation process. On the other hand, when there are one or more tiles in the frame (single_tile_in_pic_flag == 0), the decoder 410 decodes the uniform_tile_spacing_flag from the bitstream (S1030).

[0135] Decoder 410 determines whether tiles in the frame are uniformly divided based on the decoded uniform_tile_spacing_flag value (S1040). When tiles are uniformly divided (uniform_tile_spacing_flag == 1), decoder 410 decodes tile_cols_width_minus1 and tile_rows_height_minus1 from the bitstream (S1050). On the other hand, when tiles are not uniformly divided (uniform_tile_spacing_flag == 0), decoder 410 decodes num_tile_columns_minus1 and num_tile_rows_minus1 from the bitstream (S1060), and decodes tile_column_width_minus1[i] and tile_row_height_minus1[i] from the bitstream with the numbers indicated by num_tile_columns_minus1 and num_tile_rows_minus1, respectively (S1070).

[0136] The decoder 410 can divide the screen into non-uniform tiles based on the values ​​of num_tile_columns_minus1, num_tile_rows_minus1, tile_column_width_minus1[i], and tile_row_height_minus1[i], or divide the screen into uniform tiles based on the values ​​of tile_cols_width_minus1 and tile_rows_height_minus1.

[0137] Implementation Methods 1-2

[0138] Implementation methods 1-2 relate to methods for dividing a screen into squares (methods for obtaining square segmentation information). Dividing the screen into squares is only permitted in the horizontal direction. Figure 11 An example of implementation method 1-2 is shown.

[0139] The video encoding device determines whether a tile in the frame is split into multiple squares and sets the determination result as the value of the `brick_splitting_present_flag` syntax element. When `brick_splitting_present_flag` is 1, it indicates that one or more tiles in the frame are split into multiple squares, while when `brick_splitting_present_flag` is 0, it indicates that no tiles in the frame are split into multiple squares. `brick_splitting_present_flag` is encoded and signaled to the video decoding device.

[0140] The video encoding device determines the number of tiles in the frame and sets the result as the value of the `num_tiles_in_pic_minus1` syntax element. When the tiles in the frame are uniformly divided (`uniform_tile_spacing_flag == 1`) and one or more tiles in the frame are divided into multiple squares (`brick_splitting_present_flag == 1`), `num_tile_in_pic_minus1` is encoded and signaled to the video decoding device.

[0141] The video encoding device checks the height of the i-th tile (the number of CTU rows in the i-th tile, RowHeight[[i]) and sets the values ​​of the syntax elements brick_split_flag[i], uniform_brick_spacing_flag[i], brick_height_minus1[i], num_brick_rows_minus2[i], and brick_row_height_minus1[i][j] based on the result.

[0142] `brick_split_flag[i]` is a syntactic element indicating whether the i-th tile is split into squares. When the value of `RowHeight[i]` is greater than 1, `brick_split_flag[i]` can be set to "1", which indicates that the i-th tile is split into squares and encoded, and signaled to the video decoding device. This is because when the value of `RowHeight[i]` is greater than 1, horizontal tile splitting is possible. On the other hand, when the value of `RowHeight[i]` is 1 or less, `brick_split_flag[i]` is set to "0" and not signaled to the video decoding device, indicating that the i-th tile is not split into squares. This is because when the value of `RowHeight[i]` is 1 or less, horizontal tile splitting is impossible.

[0143] `uniform_brick_spacing_flag[i]` is a syntactic element indicating whether the i-th brick is uniformly divided into squares. When the value of `rowheight[i]` is greater than 2, both uniform and non-uniform division are possible. Therefore, `uniform_brick_spacing_flag[i]` can be set to "1" to indicate uniform division, or set to "0" to indicate non-uniform division. In this case, `uniform_brick_spacing_flag[i]` can be encoded and signaled to the video decoding device. On the other hand, when the value of `rowheight[i]` is 2 or less, only 1:1 uniform division is possible. Therefore, `uniform_brick_spacing_flag[i]` is set to "1" and is not signaled to the video decoding device.

[0144] `brick_height_minus1[i]` is a syntactic element indicating the value obtained by subtracting 1 from the height of a uniformly divided row of blocks, in units of CTU size. The syntactic element is not defined for the height of the bottom row of blocks within the i-th block. When blocks are uniformly divided, if the value of `rowheight[i]` exceeds 3, `brick_height_minus1[i]` is encoded and signaled to the video decoding device. Conversely, when blocks are uniformly divided and the value of `rowheight[i]` is 3, only a 1:1:1 uniform division is possible; therefore, `brick_height_minus1[i]` is set to 0 (`brick_height_minus1[i] == 0`) and is not signaled to the video decoding device.

[0145] `num_brick_rows_minus2[i]` is a syntactic element indicating the value obtained by subtracting 2 from the number of unevenly divided blocks. When the blocks are unevenly divided and the value of `RowHeight[i]` exceeds 3, `num_brick_rows_minus2[i]` is encoded and signaled to the video decoding device. On the other hand, when the blocks are unevenly divided and the value of `RowHeight[i]` is 3, only 1:2 or 2:1 uneven division is possible; therefore, `num_brick_rows_minus2[i]` is set to 0 (`num_brick_rows_minus2[i] == 0`) and is not signaled to the video decoding device.

[0146] `brick_row_height_minus1[i][j]` is a syntactic element that indicates the value obtained by subtracting 1 from the height of the j-th row of unevenly divided blocks, in CTU size. For example, when the number of unevenly divided blocks is 3 (num_brick_rows_minus2[i] == 1), `brick_row_height_minus1[i][j]` is set to the value obtained by subtracting 1 from the height of each of the two blocks. In another example, when the number of unevenly divided blocks is 5 (num_brick_rows_minus2[i] == 3), `brick_row_height_minus1[i][j]` is set to the value obtained by subtracting 1 from the height of each of the four blocks. `brick_row_height_minus1[i][j]` is encoded and signaled to the video decoding device.

[0147] Decoder 410 decodes brick_splitting_present_flag from the bitstream (S1110) and determines the value of the decoded brick_splitting_present_flag (S1120).

[0148] When the tiles in the frame are uniformly divided (uniform_tile_spacing_flag == 1) and one or more tiles in the frame are divided into multiple squares (brick_splitting_present_flag == 1), decoder 410 decodes num_tile_in_pic_minus1 from the bitstream (S1130). When the tiles in the frame are not uniformly divided (uniform_tile_spacing_flag == 0) or when no tiles in the frame are divided into multiple squares (brick_splitting_present_flag == 0), num_tile_in_pic_minus1 is not notified without a signal, and the value of num_tile_in_pic_minus1 is set to or derived as a value obtained by subtracting 1 from the number of tiles in the frame.

[0149] Decoder 410 checks the number of CTU rows (RowHeight[i]) in the i-th piece (S1140), and decodes brick_split_flag from the bitstream when RowHeight[i] > 1 (S1150). When the value of RowHeight[i] is 1 or less, no signal is given to brick_split_flag, and the value of brick_split_flag is either obtained or set to 0.

[0150] When brick_split_flag == 0, decoder 410 terminates block splitting (S1160), but when brick_split_flag == 1, decoder 410 determines the value of RowHeight[i] (S1170). When the value of RowHeight[i] is greater than 2, uniform_brick_spacing_flag is decoded from the bitstream (S1180), but when the value of RowHeight[i] is 2 or less, uniform_brick_spacing_flag is not decoded, and the value of uniform_brick_spacing_flag is set to 1 or obtained as 1.

[0151] Decoder 410 determines whether the blocks are evenly divided based on the value of uniform_brick_spacing_flag (S1182) and determines the value of RowHeight[i] (S1184, S1190). When uniform_brick_spacing_flag == 1 and the value of RowHeight[i] exceeds 3, decoder 410 decodes brick_height_minus1[i] from the bitstream (S1186). On the other hand, when uniform_brick_spacing_flag == 1 and the value of RowHeight[i] does not exceed 3, brick_height_minus1[i] is not decoded, and the value of brick_height_minus1[i] is set to or becomes 0. When uniform_brick_spacing_flag == 0 and the value of RowHeight[i] exceeds 3, decoder 410 decodes num_brick_rows_minus2[i] from the bitstream (S1192). Meanwhile, when uniform_brick_spacing_flag == 0 and the value of RowHeight[i] does not exceed 3, num_brick_rows_minus2[i] is not decoded, and the value of num_brick_rows_minus2[i] is set to or becomes 0. Decoder 410 decodes brick_row_height_minus1[i][j] from the bitstream (S1194).

[0152] As described above, the block segmentation method proposed in this disclosure is configured to signal and decode brick_height_minus1[i] and num_brick_rows_minus2[i] only when the value of RowHeight[i] exceeds 3. On the other hand, in conventional block segmentation methods, signaling and decoding brick_height_minus1[i] and num_brick_rows_minus2[i] is used for both the case where the value of RowHeight[i] exceeds 3 and the case where the value of RowHeight[i] does not exceed 3 (in other words, regardless of whether the value of rowHeight[i] exceeds 3). Therefore, compared to conventional block segmentation methods, the block segmentation method of this invention can improve bit efficiency.

[0153] Implementation Method 2

[0154] Implementation 2 relates to a method for dividing a screen into slices (a method for obtaining slice segmentation information) and a method for obtaining an enable flag.

[0155] Table 5 shows the syntactic structures associated with Implementation 2.

[0156] [Table 5]

[0157]

[0158] Implementation Method 2-1

[0159] Implementation method 2-1 relates to a method for dividing an image into slices. Slicing can be implemented by including one or more tiles / blocks. Figure 12 This illustrates the division relationship between puzzle pieces / blocks and slices. Figure 13 A method for dividing an image into slices is shown.

[0160] Figure 12 (a) shows a total of 9 tiles / squares (3×3) divided from the screen, and Figure 12 (b) to (d) show the basis Figure 12 The division relationship between the tiles / squares (solid lines) and slices (dotted lines) in (a). Figure 12 As shown in (b), a tile / block can include a slice (1:1 split), and multiple tiles / blocks can include a rectangular slice, such as... Figure 12 As shown in (c) (n:1 partition relation, where n is a natural number greater than or equal to 2). Furthermore, as... Figure 12 As shown in (d), the entire puzzle / block can form a slice (full:1 split relationship), and as... Figure 12 As shown in (e), multiple tiles / blocks can form a raster scan slice. Figure 12 In (e), the tiles / squares filled with diagonal lines in the upper right direction constitute a raster scan slice. In addition, among the tiles / squares without diagonal line patterns, the two tiles / squares at the top of the screen constitute another raster scan slice, and the other three tiles / squares at the bottom and right of the screen constitute yet another raster scan slice.

[0161] The video encoding device determines whether a slice contains one or more blocks / squares and sets the determination result as the value of the `single_brick_per_slice_flag` syntax element. When `single_brick_per_slice_flag` is 1, it indicates that the slice contains one block / square; when `single_brick_per_slice_flag` is 0, it indicates that the slice may contain one or more blocks / squares. `single_brick_per_slice_flag` is encoded and signaled to the video decoding device.

[0162] When single_brick_per_slice_flag == 1, the brick / block and slice have a 1:1 segmentation relationship; therefore, the video encoding device does not need to signal additional slice segmentation information.

[0163] The video encoding device determines the slice mode and sets the result as the value of the `rect_slice_flag` syntax element. When `rect_slice_flag` is 0, it indicates that the mode of the relevant slice is raster scan slicing mode; when `rect_slice_flag` is 1, it indicates that the mode of the relevant slice is rectangular slicing mode. When `single_brick_per_slice_flag` is 0, `rect_slice_flag` is encoded and signaled to the video decoding device.

[0164] The video encoding device determines the number of slices in the frame and sets the result as the value of the syntax element num_slices_in_pic_minus1. num_slices_in_pic_minus1 is a syntax element that indicates the value obtained by subtracting 1 from the number of slices in the frame. num_slices_in_pic_minus1 is encoded and signaled to the video decoding device.

[0165] The video encoding device determines the block / square located in the lower right part of each slice (lower right block / square) and sets the absolute value and sign of the index difference between the lower right blocks / squares as the values ​​of the bottom_right_brick_idx_delta[i] syntax element and the brick_idx_delta_sign_flag[i] syntax element.

[0166] For the second slice and subsequent (i>1) slices in the frame, bottom_right_brick_idx_delta[i] represents the absolute value of the difference between the index of the bottom-right brick / square in the previous slice and the index of the bottom-right brick / square in the subsequent slice. However, bottom_right_brick_idx_delta[i] indicates the index of the bottom-right brick / square in the first slice when it reaches the first (i==0) slice in the frame. When brick_idx_delta_sign_flag[i]==1, it indicates that the sign of bottom_right_brick_idx_delta[i] is positive; when brick_idx_delta_sign_flag[i]==0, it indicates that the sign of bottom_right_brick_idx_delta[i] is negative. The bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] are encoded and signaled to the video decoding device.

[0167] Before signaling bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i], the bottom_right_brick_idx_length_minus1 syntax element is encoded and signaled. bottom_right_brick_idx_length_minus1 indicates the number of bits used to represent bottom_right_brick_idx_delta[i].

[0168] Decoder 410 decodes single_brick_per_slice_flag from the bitstream (S1310). When single_brick_per_slice_flag does not exist in the bitstream, the value of single_brick_per_slice_flag is set to or obtained as "1".

[0169] Decoder 410 determines whether to decode (obtain) additional slice segmentation information based on the value of single_brick_per_slice_flag (S1320). When single_brick_per_slice_flag == 1, the additional slice segmentation information is not decoded, and the slice segmentation information is derived from the tile / block segmentation information. Conversely, when single_brick_per_slice_flag == 0, decoder 410 decodes rect_slice_flag from the bitstream (S1330). When rect_slice_flag does not exist in the bitstream, the value of rect_slice_flag is set to or obtained as "1".

[0170] When the slice mode is rectangular slice mode (rect_slice_flag == 1) and the slice includes one or more tiles / blocks (single_brick_per_slice_flag == 0) (S1340), the decoder 410 decodes num_slices_in_pic_minus1 and bottom_right_brick_idx_length_minus1 from the bitstream (S1350). When num_slices_in_pic_minus1 does not exist in the bitstream and single_brick_per_slice_flag == 1, num_slices_in_pic_minus1 is not decoded and is set to or derived as a value obtained by subtracting 1 from the number of tiles / blocks in the picture.

[0171] When the slice mode is rectangular slice mode (rect_slice_flag == 1) and the slice includes one or more tiles / blocks (single_brick_per_slice_flag == 0) (S1340), decoder 410 decodes bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] from the bitstream (S1360). Since bottom_right_brick_idx_delta[i] and brick_idx_delta_sign_flag[i] are not signaled (decoded) when single_brick_per_slice_flag == 1, bottom_right_brick_idx_delta[i] is set to or becomes 1 (bottom_right_brick_idx_delta[i] == 1).

[0172] Table 6 shows the applications... Figure 12 The slicing method of implementation 2-1, which describes the segmentation relationship between the blocks and slices.

[0173] [Table 6]

[0174]

[0175] exist Figure 12 In case (b), since the segmentation relationship between the tile / block and the slice is the same (1:1 relationship), the signal is used to notify by setting single_brick_per_slice_flag to "1"; other slice segmentation information is obtained based on the tile / block segmentation information without signal notification.

[0176] exist Figure 12 In case (c), since the segmentation relationship between the tiles / blocks and slices is different, a signal is used to notify by setting single_brick_per_slice_flag to "0"; since the slice mode is rectangular slice mode, a signal is used to notify by setting rect_slice_flag to 1. In addition, since the number of slices in the picture is 3, a signal is used to notify by setting num_slices_in_pic_minus1 to 2; bottom_right_brick_idx_delta[i] is set and signaled so that bottom_right_brick_idx_delta[0] == 2, bottom_right_brick_idx_delta[1] == 3, and bottom_right_brick[2] == 3. In addition, brick_idx_delta_sign_flag[i] is set and signaled, such that brick_idx_delta_sign_flag[0] == 1 (+), brick_idx_delta_sign_flag[1] == 1, and brick_idx_delta_sign_flag[2] == 1.

[0177] exist Figure 12 In cases (d) and (e), since the segmentation relationship between the tiles / blocks and slices is different, a signal is used to notify the user by setting `single_brick_per_slice_flag` to "0"; since the slicing mode is raster scan slicing mode, a signal is used to notify the user by setting `rect_slice_flag` to 0. The remaining slice segmentation information is not notified by a signal.

[0178] In different implementations Figure 12The slice mode of (d) can be set to rectangular slice mode (rect_slice_flag == 1), and the number of slices in the image can be set to 1 (num_slices_in_pic_minus1 == 0). In this case, Table 7 shows an example of slice segmentation information notified by a signal.

[0179] [Table 7]

[0180] Figure 12 of (d) single_brick_per_slice_flag 0 rect_slice_flag 1 num_slices_in_pic_minus1 0 bottom_right_brick_idx_length_minus1 0 bottom_right_brick_idx_delta - brick_idx_delta_sign_flag -

[0181] Implementation Method 2-2

[0182] Implementation 2-2 relates to a method for obtaining an enable flag (loop_filter_across_slices_enabled_flag) that indicates whether in-loop filtering is applied to the boundaries between slices.

[0183] Whether to signal the `loop_filter_across_slices_enabled_flag` can be determined based on preset conditions. This is because, depending on the segmentation relationship between tiles / squares and slices, the boundaries of tiles / squares and slices can partially or completely overlap (or be identical). For example, in Figure 12 In case (b), the boundaries of the puzzle / block and the slice are exactly the same; Figure 12 In case (c), the boundaries of the puzzle / block and the boundaries of the slice are partially the same; Figure 12 In case (d), the slice boundary becomes the image boundary, and... Figure 12 In case (e), the boundaries of the tiles / blocks and the boundaries of the slices can be partially the same.

[0184] exist Figures 14 to 17 Various examples of implementation methods 2-2 are shown in the figure.

[0185] The video encoding device encodes the slice segmentation information and sends the encoded information to the video decoding device via a signal. The decoder 410 decodes the slice segmentation information (S1410, S1510, S1610, S1710) from the bitstream and divides the image into one or more slices based on the slice segmentation information.

[0186] Depending on preset conditions, the video encoding device determines whether to signal `loop_filter_across_slices_enabled_flag`. In other words, when the preset conditions are met, `loop_filter_across_slices_enabled_flag` is encoded and signaled to the video decoding device. Decoder 410 determines whether the preset conditions are met (S1420, S1520, S1620, S1720), and decodes `loop_filter_across_slices_enabled_flag` from the bitstream based on the determination result (S1430, S1530, S1630, S1730).

[0187] The preset conditions may include one or more conditions such as whether there are multiple slices in the image (whether the image is divided into multiple slices) and whether the slice mode is a rectangular slice mode.

[0188] For example, such as Figure 14 As shown, the preset conditions may include whether there are multiple slices in the frame (num_slices_in_pic_minus1) and whether the slice mode is rectangular slice mode (rect_slice_flag). In this case, when there are multiple slices in the frame or the slice mode is not rectangular slice mode (num_slices_in_pic_minus1>0||rect_slice_flag==0), a signal is sent to notify and decode loop_filter_across_slices_enabled_flag.

[0189] exist Figure 12 In cases (b) and (c), since num_slices_in_pic_minus1 > 0, the loop_filter_across_slices_enabled_flag is signaled and decoded, and... Figure 12 In cases (d) and (e), since rect_slice_flag == 0, loop_filter_across_slices_enabled_flag is signaled and decoded.

[0190] In different implementations, the preset conditions may include whether the slicing mode is a rectangular slicing mode, whether the slice includes one or more tiles / blocks, and whether one or more conditions are applied to the boundaries between tiles / blocks with in-loop filtering.

[0191] For example, such as Figure 15As shown, the preset conditions can include all of the following: whether the slice mode is a rectangular slice mode (rect_slice_flag), whether the slice includes one or more tiles / blocks (single_brick_per_slice_flag), and whether in-loop filtering is applied to the boundaries between tiles / blocks (loop_filter_across_bricks_enabled_flag).

[0192] In this case, when the slicing mode is rectangular slicing mode, the slice consists of one or more tiles / blocks, and in-loop filtering is applied to the boundaries between tiles / blocks (rect_slice_flag == 1 && single_brick_per_slice_flag == 0 && loop_filter_across_bricks_across_bricks_across_bricks_enable_flag == 1), and loop_filter_across_slices_enabled_flag is signaled and decoded. Table 5 shows... Figure 15 The syntactic structure of the preset conditions is shown.

[0193] exist Figure 12 In case (b), since single_brick_per_slice_flag == 1, loop_filter_across_slices_enabled_flag is not signaled or decoded. Because the partitioning relationship between tiles / blocks and slices is 1:1 (i.e., all boundaries of tiles / blocks coincide with the boundaries of slices), it is not necessary to separately signal loop_filter_across_slices_enabled_flag for slice boundaries.

[0194] exist Figure 12 In case (c), since rect_slice_flag == 1, single_brick_per_slice_flag == 0, and loop_filter_across_bricks_enabled_flag == 1, loop_filter_across_slices_enabled_flag is signaled. Figure 12 In cases (d) and (e), since rect_slice_flag == 0, no signal is needed to notify loop_filter_across_slices_enabled_flag. Here, in cases such as... Figure 12In the case of the full :1 split relationship shown in (d), the slice boundary is consistent with the image boundary, so there is no need to separately signal the loop_filter_across_slices_enabled_flag for the slice boundary.

[0195] In different implementations, such as Figure 16 As shown, the preset conditions may include whether one or more tiles / blocks are included in the slice (single_brick_per_slice_flag) and whether in-loop filtering is applied to the boundaries between tiles / blocks (loop_filter_across_bricks_enabled_flag).

[0196] In this case, when one or more tiles / blocks are included in a slice, and in-loop filtering is applied to the boundaries between tiles / blocks (single_brick_per_slice_flag == 0 &&&loop_filter_across_bricks_enabled_flag == 1), loop_filter_across_slices_enabled_flag is signaled and decoded.

[0197] Table 8 shows Figure 16 The syntactic structure of the preset conditions is shown.

[0198] [Table 8]

[0199]

[0200] exist Figure 12 In case (b), since single_brick_per_slice_flag == 1, no signal is needed to notify loop_filter_across_slices_enabled_flag. Figure 12 In cases (c) to (e), since single_brick_per_slice_flag == 0 and loop_filter_across_bricks_enabled_flag == 1, loop_filter_across_slices_enabled_flag is signaled.

[0201] In different implementations, such as Figure 17As shown, the preset conditions may include whether there are multiple slices in the image (num_slices_in_pic_minus1), whether the slicing mode is rectangular slicing mode (rect_slice_flag), whether the slice includes one or more bricks / blocks (single_brick_per_slice_flag), and whether in-loop filtering is applied to the boundaries between bricks / blocks (loop_filter_across_bricks_enabled_flag).

[0202] In this scenario, when in-loop filtering is applied to the boundaries between tiles / blocks, one or more tiles / blocks are included in a slice, and the slice mode is not a rectangular slice mode (loop_filter_across_bricks_enabled_flag == 1 && single_brick_per_slice_flag == 0 && rect_slice_flag == 0), loop_filter_across_slice_enabled_flag is signaled and decoded.

[0203] Alternatively, when in-loop filtering is applied to the boundaries between tiles / blocks, one or more tiles / blocks are included in a slice, the slice mode is rectangular slice mode, and the number of slices in the image is multiple (loop_filter_across_bricks_enabled_flag == 1 && single_brick_per_slice_flag == 0 && rect_slice_flag == 1 && num_slices_in_pic_minus1 > 0), loop_filter_across_slices_enabled_flag is signaled and decoded.

[0204] Table 9 shows... Figure 17 The syntactic structure of the preset conditions is shown.

[0205] [Table 9]

[0206]

[0207] If the slice settings shown in Table 7 are applied Figure 17 Based on the preset conditions, the following can be used to determine whether to use a signal to notify the enable flag. Figure 12 In case (b), since single_brick_per_slice_flag == 1, no signal is needed to notify loop_filter_across_slices_enabled_flag. Figure 12 In cases (c) and (e), since loop_filter_across_bricks_enabled_flag == 1, single_brick_per_slice_flag == 0, rect_slice_flag == 1, and num_slices_in_pic_minus1 > 0, loop_filter_across_slices_enabled_flag is signaled. Figure 12 In case (d), since num_slices_in_pic_minus1 == 0, no signal is needed to notify loop_filter_across_slices_enabled_flag.

[0208] Implementation Method 3

[0209] Implementation method 3 relates to a method for dividing a screen into sub-screens (a method for obtaining sub-screen segmentation information). Figure 18 An example of implementation method 3 is shown.

[0210] After dividing the frame into multiple CTUs of predetermined size, the video encoding device encodes the size information of the CTUs (CTU size information) and signals this encoded information to the video decoding device. Furthermore, after dividing the CTUs into CUs based on a tree structure, the video encoding device encodes the minimum size information of the CUs (MinCb size information) and signals this encoded information to the video decoding device. The MinCb size information can be defined and signaled at one or more locations at the SPS level, PPS level, and slice header level of the bitstream.

[0211] The video encoding device divides the screen into multiple sub-screens. The sub-screens can be based on the CTU size derived from CTU size information or on the MinCb size derived from MinCb size information.

[0212] Therefore, sub-screen segmentation information can represent sub-screens within a frame in units of CTU size or MinCb size. Sub-screen segmentation information can include sub-screen position information indicating the location of each sub-screen and sub-screen size information indicating the size of each sub-screen. Sub-screen position information can represent the location of a sub-screen within a frame in units of CTU size or MinCb size. Sub-screen size information can also represent the size of a sub-screen in units of CTU size or MinCb size.

[0213] Decoder 410 decodes CTU size information and MinCb size information from the bitstream (S1810). Furthermore, decoder 410 decodes sub-picture segmentation information from the bitstream (S1820).

[0214] The decoder 410 divides the screen into CTUs based on the CTU size information and divides the screen into sub-screens based on the sub-screen division information (S1830).

[0215] Traditional sub-screen segmentation methods represent sub-screens using units of 4 samples. In other words, traditional sub-screen segmentation methods use multiples of 4 as the unit for sub-screen segmentation. However, as mentioned above, a screen can be segmented into multiple CTUs, into tiles / blocks containing one or more CTUs, into slices containing one or more tiles / blocks, and into sub-screens containing one or more slices. Therefore, representing tiles / blocks, slices, and sub-screens in units of CTU size (multiples of CTU) can be the most efficient. Therefore, when sub-screens are represented in units of CTU size according to Embodiment 3, efficient sub-screen segmentation can be achieved.

[0216] Based on this implementation, sub-screen segmentation information can represent sub-screens directly or indirectly in units of CTU size or MinCb size. Here, the indirect representation method connects the CTU and the sub-screen through different sub-units. Different sub-units can be grids, and the grid can be represented in units of CTU size. The sub-screen is represented in units of grid size, and consequently, the sub-screen can be represented in units of CTU size.

[0217] Figure 19 An example of indirect representation is shown, and Table 10 shows the syntactic structure of indirect representation.

[0218] [Table 10]

[0219]

[0220] The video encoding device determines whether to perform subpics segmentation (whether subpics parameters exist in the bitstream) and sets the determination result as the value of the subpics_present_flag syntax element. When subpics_present_flag == 1, it indicates that subpics segmentation should be performed, while when subpics_present_flag == 0, it indicates that subpics segmentation should not be performed.

[0221] When subpics_present_flag == 1, the video encoding device decodes max_subpics_minus1 and subpic_grid_row_height_minus1, where max_subpics_minus1 is a syntax element indicating the maximum number of subpics, subpic_grid_col_width_minus1 is a syntax element indicating the width of a grid column in units of CTU size or MinCb size, and subpic_grid_row_height_minus1 is a syntax element indicating the height of a grid row in units of CTU size or MinCb size. The encoded syntax elements are then signaled to the video decoding device.

[0222] `max_subpics_minus1` represents the value obtained by subtracting 1 from the maximum number of subpics, `subpic_grid_col_width_minus1` represents the value obtained by subtracting 1 from the width of the grid column, and `subpic_grid_row_height_minus1` represents the value obtained by subtracting 1 from the height of the grid row.

[0223] The video encoding device encodes the subpic index (subpic_grid_idx[i][j]) and signals the encoded subpic index to the video decoding device. The subpic index indicates the index of the subpic to which each grid belongs, and it can be used to determine which grid belongs to which subpic.

[0224] Decoder 410 decodes subpics_present_flag from the bitstream (S1910) and determines the value of subpics_present_flag (S1920).

[0225] When subpics_present_flag == 1, decoder 410 decodes max_subpics_minus1, subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1 from the bitstream (S1930).

[0226] Since the grid has a fixed width and height, the number of grid columns (i) can be determined based on subpic_grid_col_width_minus1, and the number of grid rows (j) can be determined based on subpic_grid_row_height_minus1. As a result, the screen can be divided into grids based on subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1.

[0227] Decoder 410 can decode the sub-picture index from the bitstream (S1940) and divide the picture into sub-pictures by determining the grids assigned the same sub-picture index from multiple grids as sub-pictures.

[0228] While exemplary embodiments have been described for illustrative purposes, those skilled in the art will understand that various modifications and changes are possible without departing from the spirit and scope of the embodiments. For the sake of brevity and clarity, exemplary embodiments have been described. Therefore, those skilled in the art will understand that the scope of the embodiments is not limited to those explicitly described above, but includes the claims and their equivalents.

[0229] Cross-references to related applications

[0230] This application claims priority to Korean Patent Applications No. 10-2019-0119353 and No. 10-2020-0113632, filed on September 27, 2019 and September 7, 2020, respectively, the entire contents of which are incorporated herein by reference.

Claims

1. A video decoding method, comprising the following steps: Decode information about the subunits of the frame from the bitstream; as well as The image is reconstructed based on the information about the sub-unit. The step of decoding the information about the sub-unit includes the following steps: Decode the CTU size information from the bitstream, indicating the size of the Code Tree Unit (CTU) within the frame. Decoding sub-picture segmentation information from the bitstream, the sub-picture segmentation information being used to divide the picture into one or more sub-pictures, each of the one or more sub-pictures consisting of at least one slice, wherein the sub-picture segmentation information is defined in units of CTU size. Decode the tile segmentation information from the bitstream regarding dividing the image into one or more tiles. Decode from the bitstream slice segmentation information about dividing the image into one or more slices, and Based on the slice pattern among multiple slice patterns including rectangular slice pattern and raster scan slice pattern, a bitstream decoding enable flag is generated, which indicates whether filtering is performed at the boundary of the slice.

2. The video decoding method according to claim 1, wherein, The sub-screen segmentation information includes sub-screen size information, which represents the size of each sub-screen in units of the CTU size.

3. The video decoding method according to claim 1, wherein, The slice segmentation information is defined based on the pieces determined by the piece segmentation information.

4. The video decoding method according to claim 3, wherein, When there are multiple slices and the slice mode is the rectangular slice mode, or when the slice mode is the raster scan slice mode, the enable flag is decoded.

5. A video encoding method, comprising the following steps: The image is divided into sub-units for encoding; as well as Information about the subunit is encoded; The step of encoding the information about the sub-unit includes: The CTU size information, which indicates the size of the coding tree unit (CTU) within the image, is encoded. The sub-screen segmentation information is encoded to divide the screen into one or more sub-screens, each of which consists of at least one slice. The sub-screen segmentation information is defined in units of CTU size. Encode the tile segmentation information that divides the image into one or more tiles. Encode the slice segmentation information that divides the image into one or more slices, and Based on the slice pattern among multiple slice patterns, including rectangular slice pattern and raster scan slice pattern, an enable flag is encoded, which indicates whether filtering is performed at the boundary of the slice.

6. An apparatus for transmitting a bitstream comprising encoded video data, the apparatus comprising at least one processor configured to: The bitstream is generated by dividing the frame into sub-units for encoding and encoding information about the sub-units; and The bitstream is sent to the video decoding device. in, The at least one processor is configured to encode the information about the subunit by: The CTU size information, which indicates the size of the coding tree unit (CTU) within the image, is encoded. The sub-screen segmentation information is encoded to divide the screen into one or more sub-screens, each of which consists of at least one slice. The sub-screen segmentation information is defined in units of CTU size. Encode the tile segmentation information that divides the image into one or more tiles. Encode the slice segmentation information that divides the image into one or more slices, and Based on the slice pattern among multiple slice patterns, including rectangular slice pattern and raster scan slice pattern, an enable flag is encoded, which indicates whether filtering is performed at the boundary of the slice.

Images