Method and device for sending slice segmentation information in image and video encoding and decoding

Abstract

A method and apparatus for sending or parsing image segmentation information are disclosed. According to the method, a current image is segmented into one or more slices and one or more tiles according to the image segmentation information. A control syntax is determined, and unless the image segmentation information indicates that a rectangular slice mode is selected and each sub-image is allowed to include more than one rectangular slice, and the current image includes only one rectangular slice in the current image, the control syntax is sent from a video bitstream on an encoder side or parsed from a video bitstream on a decoder side. If the image segmentation information indicates that there are multiple slices in the current image and the control syntax indicates that loop filtering is enabled, loop filtering is applied across slice boundaries.

Description

[0001] Cross-references

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 003,362, filed April 1, 2020, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present invention generally relates to picture partitioning in video coding systems and more particularly to signaling of partitioning information associated with partitioning a picture into slices and tiles. Background Art

[0004] High-efficiency video coding (HEVC) is the latest international video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC) (Rec. ITU-T H.265ISO / IEC23008-2version 3: High Efficiency Video Coding, April, 2015). Figure 1 A block diagram of a HEVC codec system is provided. An input video signal is predicted using inter / intra prediction (110) by deriving a prediction signal (136) from an already coded image region. The prediction residual signal (116) is processed by a linear transform (transform, T) 118. The transform coefficients are quantized (quantized, Q) 120 and entropy encoded in a bitstream by an entropy encoder (122) along with other auxiliary information. After inverse transform (IT) 126 of the inverse quantization (IQ) 124 transform coefficients, a reconstructed signal (in Figure 1 The reconstructed signal is further processed by loop filtering (e.g., de-blocking filter (DF) 130 and non-deblocking filter (NDF) 131) to remove coding artifacts. The decoded images are stored in a reference image buffer (134) for predicting future images in the input video signal.

[0005] In HEVC, a coded image is partitioned into non-overlapping square block regions represented by associated coding tree units (CTUs). A coded image can be represented by a set of slices, each slice comprising an integer number of CTUs. A single CTU in a slice is processed in raster scanning order. Bi-predictive (B) slices can be decoded using intra prediction or inter prediction, which uses up to two motion vectors and reference indices to predict the sample values ​​of each block. Predictive (P) slices can be decoded using intra prediction or inter prediction, which uses up to one motion vector and reference index to predict the sample values ​​of each block. Intra (I) slices are decoded using only intra prediction.

[0006] A CTU can be partitioned into multiple non-overlapping coding units (CUs) using a recursive quadtree (QT) structure to accommodate various local motion and texture characteristics. One or more prediction units (PUs) are specified for each CU. The prediction unit, together with the associated CU syntax, serves as the basic unit for sending prediction sub-information. The specified prediction process is used to predict the values ​​of related pixel samples within the PU. The CU can be further partitioned using a residual quadtree (RQT) structure to represent the related prediction residual signal. The leaf nodes of the RQT correspond to transform units (TUs). The transform unit includes a transform block of luminance samples of size 8×8, 16×16 or 32×32 or a transform block of luminance samples of size 4×4, and two corresponding transform blocks of chrominance samples of the image in the 4:2:0 color format. Integer transforms are applied to the transform blocks and the hierarchical values ​​of the quantized coefficients and other auxiliary information are entropy encoded in the bitstream. Figure 2 An example of a block partition 210 (left) and its corresponding QT representation 220 (right) is shown. Solid lines represent CU boundaries and dashed lines represent TU boundaries.

[0007] The terms coding tree block (CTB), coding block (CB), prediction block (PB), and transform block (TB) are defined to specify a 2-D sample array of one color component associated with a CTU, CU, PU, ​​and TU, respectively. Thus, a CTU consists of one luma CTB, two chroma CTBs, and associated syntax elements. Similar relationships are valid for CUs, PUs, and TUs. Tree partitioning is usually applied to both luma and chroma, but some exceptions may apply when chroma reaches certain minimum sizes.

[0008] ITU-T SG16 WP3 and JVET of ISO / IEC JTC1 / SC29 / WG11 are currently establishing the next generation video codec standard. Some promising new coding tools have been adopted in the Versatile Video Coding (VVC) draft of JVET-Q2001 (B. Bross J. Chen, S. Liu, “Versatile Video Coding (Draft 8),” Document of Joint Video Experts Team of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, JVET-Q2001, 17th Meeting: Brussels, BE, 7–17 January 2020). In the VVC draft specified in JVET-Q2001, similar to HEVC, the coded image has been partitioned into non-overlapping square block regions represented by CTUs. Each CTU can be partitioned into one or more smaller CUs by nested multi-type trees with quadtrees and binary and ternary splits. The resulting CU partition may be square or rectangular.

[0009] In the VVC draft specified in JVET-Q2001, a tile is a sequence of CTUs covering a rectangular area of ​​an image. The CTUs in a tile are scanned in a raster scan order. An image is divided into one or more tile rows and one or more tile columns. A slice consists of an integer number of complete tiles or an integer number of consecutive complete CTU columns in a tile of an image. Two slicing modes are supported, namely raster scan slicing mode and rectangular slicing mode, as indicated by the syntax element rect_slice_flag. In raster scan slicing mode, a slice includes a complete sequence of slices in a slice raster scan of an image. In rectangular slicing mode, a slice includes multiple complete tiles that together form a rectangular area of ​​an image, or includes multiple consecutive complete columns of a tile that together form a rectangular area of ​​an image. Within the rectangular area corresponding to the slice, the tiles in the rectangular slice are scanned in a slice raster scan order. Figure 3 and Figure 4 Examples are provided for segmenting an image into tiles and slices in raster scan tiling mode and rectangular tiling mode, respectively. Figure 3 An example of segmenting an image with 18x12 luma CTUs into 12 tiles and 3 raster scan slices is shown. Each CTU is represented by a small rectangle surrounded by dashed lines, each tile is represented by a thin solid line, and each slice is represented by a gray area surrounded by thick lines. Figure 4 An example of partitioning an image with 18x12 luma CTUs into 24 tiles and 9 rectangular slices is shown. The syntax element slice_address specifies the raster scan tile index of the first tile in the slice. The value of the syntax element num_tiles_in_slice_minus1 plus 1 specifies the number of contiguous tiles in the current slice. Summary of the invention

[0010] A method and apparatus for sending or parsing image segmentation information are disclosed. According to the method, a current image is segmented into one or more slices and one or more tiles according to the image segmentation information. A control syntax is determined, and unless the image segmentation information indicates that a rectangular slice mode is selected and each sub-image is allowed to include more than one rectangular slice, and the current image includes only one rectangular slice in the current image, the control syntax is sent from a video bitstream on the encoder side or parsed from a video bitstream on the decoder side. If the image segmentation information indicates that multiple slices exist in the current image and the control syntax indicates that in-loop filtering is enabled, in-loop filtering is applied across slice boundaries. The syntax is signaled or parsed from a slice header level of a video bitstream corresponding to a target slice. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1An exemplary adaptive inter / intra video coding system is shown.

[0012] Figure 2 An example of block partitioning is shown, where the result of the block partitioning is shown on the left and the codec tree (also called partition tree structure) is shown on the right.

[0013] Figure 3 An example of segmenting an image with 18x12 luma CTUs into 12 tiles and 3 raster scan slices is shown.

[0014] Figure 4 An example of segmenting an image with 18x12 luma CTU into 24 tiles and 9 rectangular slices is shown.

[0015] Figure 5 A flow chart of an exemplary video decoding system according to an embodiment of the present invention is shown, in which a control syntax for controlling whether a loop filter is applied across a slice boundary is parsed if a set of conditions are met. DETAILED DESCRIPTION

[0016] The following description is the best mode of implementing the present invention. The description is intended to illustrate the general principles of the present invention and should not be considered as limiting. The scope of the present invention is best determined by reference to the attached claims.

[0017] In the VVC draft specified in JVET-Q2001, a tile is a sequence of CTUs covering a rectangular area of ​​an image. The CTUs in a tile are scanned in a raster scan order. An image is partitioned into one or more tile rows and one or more tile columns. A slice consists of an integer number of complete tiles or an integer number of consecutive complete CTU columns in a tile of an image. When the coded image is further partitioned into more than one slice or one tile using the syntax element no_pic_partition_flag equal to 0, the information used to derive the tile partitioning of the coded image is signaled by the syntax elements pps_log2_ctu_size_minus5, num_exp_tile_columns_minus1, num_exp_tile_rows_minus1, and tile_row_height_minus1 in the picture parameter set (PPS). When rectangular slice mode is used (i.e., rect_slice_flag = 1), the syntax element tile_idx_delta_present_flag may be signaled to specify whether the coded picture is partitioned into rectangular slice rows and rectangular slice columns in slice raster order. tile_idx_delta_present_flag equal to 0 indicates that the tile_idx_delta[i] syntax element is not in the PPS, and all pictures referencing the PPS are partitioned into rectangular slice rows and rectangular slice column rows in slice raster order. tile_idx_delta_present_flag equal to 1 indicates that the tile_idx_delta[i] syntax element (indicating the difference between the tile index of the current slice and the tile index of the next slice) may be present in the PPS, and all rectangular slices in the pictures referencing the PPS are specified in the order indicated by the value of tile_idx_delta[i], with increasing values ​​of i. When the number of slices in the coded picture is equal to 1, the value of tile_idx_delta_present_flag is not sent, and is inferred to be equal to 0.

[0018] According to an aspect of the invention, when the number of tiles in a coded picture is equal to 1, there is only one tile index equal to 0 in the coded picture and tile_idx_delta is not sent. In mode 1, sending tile_idx_delta_present_flag further depends on the number of tiles in the coded picture. In one embodiment, tile_idx_delta_present_flag is signaled only when the number of slices in the coded picture is greater than 1 and the number of tiles in the coded picture is greater than an integer threshold T, where T can be equal to 1, 2 or 3. When tile_idx_delta_present_flag is not sent, it is inferred to be equal to 0.

[0019] According to another aspect of the present invention, when the number of tile rows or tile columns in a coded image is equal to 1, the coded image is always partitioned into rectangular slice rows and rectangular slice columns (vertically or horizontally) according to a slice raster order.

[0020] In mode 2, sending tile_idx_delta_present_flag further depends on whether the number of tile rows or tile columns in the coded picture is greater than 1. In one embodiment, tile_idx_delta_present_flag is signaled only when the number of slices in the coded picture is greater than 1 and the number of tile rows or tile columns in the coded picture is greater than 1. When tile_idx_delta_present_flag is not sent, it is inferred to be equal to 0.

[0021] According to another aspect of the invention, a syntax control flag specifying whether loop filtering operations should be applied across slice boundaries is only relevant for coded pictures comprising more than one slice.In the proposed method, signaling of the control flag depends on whether the number of slices in the coded picture is greater than one.

[0022] In an example embodiment, according to the present invention, the video encoder specified in JVET-Q2001 is modified using method 2 of sending tile_idx_delta_present_flag. The modified syntax table for PPS is provided in Table 1. In the proposed method, tile_idx_delta_present_flag is signaled only when the number of slices in the coded picture is greater than 1 and the number of tile rows or the number of tile columns in the coded picture is greater than 1. When rectangular slice mode is used and the number of coded slices is equal to 1, the syntax element loop_filter_across_slices_enabled_flag is not signaled, which specifies whether loop filtering operations can be performed across slice boundaries in pictures referring to the PPS.

[0023] Table 1 Syntax table modified based on JVET-Q2001 according to the proposed method

[0024]

[0025]

[0026]

[0027] Any of the methods proposed above can be implemented in an encoder and / or decoder. For example, any of the methods proposed above can be implemented in a high-level syntax encoding module, a high-level syntax decoding module of an encoder and / or a decoder. Alternatively, any of the methods proposed above can be implemented as a circuit integrated into a high-level syntax encoding module of an encoder and / or a high-level syntax decoding module of a decoder. Any of the methods proposed above can also be implemented in an image encoder and / or a decoder, wherein the resulting bitstream corresponds to a coded frame using only intra-image prediction.

[0028] Figure 5 A flow chart of an exemplary video decoding system according to an embodiment of the present invention is shown, in which a control syntax for controlling whether a loop filter is applied across a slice boundary is parsed if a set of conditions are met. The steps shown in the flowchart may be implemented as program codes that can be executed on one or more processors (e.g., one or more central processing units (CPUs)) at the encoder side and / or the decoder side. The steps shown in the flowchart may also be implemented based on hardware, such as one or more electronic devices or processors arranged to perform the steps in the flowchart. According to the method, in step 510, a video bitstream including a current image is received. In step 520, image segmentation information is parsed from the video bitstream, wherein the current image is segmented into one or more slices and one or more tiles according to the image segmentation information. In step 530, a control syntax is determined, and unless the image segmentation information indicates that a rectangular slice mode is selected and each sub-image is allowed to include more than one rectangular slice, and the current image includes only one rectangular slice in the current image, the control syntax is parsed from the video bitstream. In step 540, a reconstructed image is derived from the video bitstream, wherein the reconstructed image includes the image segmentation according to the image segmentation information. In step 550, if the image segmentation information indicates that there are multiple slices in the current image and the control syntax indicates that loop filtering is enabled, loop filtering is applied across slice boundaries.

[0029] It can be concluded accordingly that Figure 5 Flowchart of an exemplary video encoding system corresponding to the decoder in.

[0030] The flowchart shown is intended to illustrate an example of video coding according to the present invention. Those skilled in the art may modify each step, rearrange the steps, split the steps or combine the steps to practice the present invention without departing from the spirit of the present invention. In the present invention, specific syntax and semantics have been used to illustrate examples for implementing embodiments of the present invention. Those skilled in the art may implement the present invention by replacing syntax and semantics with equivalent syntax and semantics without departing from the spirit of the present invention.

[0031] The above description is presented to enable those skilled in the art to implement the present invention provided in the context of a specific application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but to be consistent with the broadest range consistent with the principles and novel features disclosed herein. In the above detailed description, various specific details are shown in order to provide a thorough understanding of the present invention. However, those skilled in the art will appreciate that the present invention may be implemented.

[0032] The embodiments of the present invention as described above can be implemented in various hardware, software codes or a combination of the two. For example, the embodiments of the present invention can be integrated into one or more circuits of a video compression chip, or integrated into video compression software to perform the program code of the processing described herein. The embodiments of the present invention can also be program codes to be executed on a digital signal processor (DSP) to perform the processing described herein. The present invention can also include many functions performed by a computer processor, a digital signal processor, a microprocessor or a field programmable gate array (FPGA). These processors can be configured to perform specific tasks according to the present invention by executing machine-readable software codes or firmware codes that define the specific methods embodied in the present invention. Software codes or firmware codes can be developed in different programming languages ​​and different formats or styles. Software codes can also be compiled for different target platforms. However, different code formats, styles and languages ​​of software codes and other means of configuring codes to perform tasks according to the present invention will not depart from the spirit and scope of the present invention.

[0033] The present invention may be implemented in other specific forms without departing from the spirit or essential features of the present invention. The described examples should be considered in all respects only as illustrative and not restrictive. Therefore, the scope of the present invention is indicated by the appended claims rather than the foregoing description. All changes falling within the equivalent meaning and scope of the scope of the claimed protection should be included within its scope.

Claims

1. A video sequence decoding method, the method comprising: receiving a video bitstream including a current image; Parsing image segmentation information from the video bitstream, wherein the current image is segmented into one or more slices and one or more tiles according to the image segmentation information; determining a control syntax, the control syntax being parsed from the video bitstream unless the picture segmentation information indicates that a rectangular slice mode is selected, allowing each sub-picture to include more than one rectangular slice, and the current picture includes only one rectangular slice; deriving a reconstructed image from the video bitstream, wherein the reconstructed image includes an image segmentation according to the image segmentation information; and If the image segmentation information indicates that there are multiple slices in the current image, and the control syntax indicates that cross-slice boundary loop filtering is enabled, loop filtering is applied across slice boundaries.

2. The video sequence decoding method according to claim 1, characterized in that: The control syntax is at the picture parameter set level of the video bitstream.

3. The video sequence decoding method according to claim 1, characterized in that: If the image segmentation information indicates that the rectangular slice mode is selected, it is inferred that the control syntax has a value indicating no loop filtering across slice boundaries, allowing each sub-image to include more than one rectangular slice, and the current image includes only one rectangular slice.

4. An apparatus for decoding a video sequence, the apparatus comprising one or more electronic circuits or processors, the one or more electronic circuits or processors being configured to: receiving a video bitstream including a current image; Parsing image segmentation information from the video bitstream, wherein: dividing the current image into one or more slices and one or more blocks according to the image segmentation information; determining a control syntax, the control syntax being parsed from the video bitstream unless the picture segmentation information indicates that a rectangular slice mode is selected, allowing each sub-picture to include more than one rectangular slice, and the current picture includes only one rectangular slice; deriving a reconstructed image from the video bitstream, wherein the reconstructed image includes an image segmentation according to the image segmentation information; as well as If the image segmentation information indicates that there are multiple slices in the current image, and the control syntax indicates that cross-slice boundary loop filtering is enabled, loop filtering is applied across slice boundaries.

5. A video sequence encoding method, the method comprising: receiving input data corresponding to a current image; Splitting the current image into one or more slices and / or one or more tiles; Sending image segmentation information in the video bitstream; determining a control syntax, the control syntax being sent from the video bitstream unless the picture segmentation information indicates that a rectangular slice mode is selected, allowing each sub-picture to include more than one rectangular slice, and the current picture includes only one rectangular slice; as well as If there is more than one slice in the current picture and the control syntax indicates that cross-slice boundary loop filtering is enabled, loop filtering is applied across slice boundaries.

6. The video sequence encoding method according to claim 5, characterized in that: The control syntax is at the picture parameter set level of the video bitstream.

7. The video sequence encoding method according to claim 5, characterized in that: If the image segmentation information indicates that the rectangular slice mode is selected, it is inferred that the control syntax has a value indicating no loop filtering across slice boundaries, allowing each sub-image to include more than one rectangular slice, and the current image includes only one rectangular slice.

8. An apparatus for encoding a video sequence, the apparatus comprising one or more electronic circuits or processors, the one or more electronic circuits or processors being configured to: receiving input data corresponding to a current image; Splitting the current image into one or more slices and / or one or more tiles; Sending image segmentation information in the video bitstream; determining a control syntax, the control syntax being sent from the video bitstream unless the picture segmentation information indicates that a rectangular slice mode is selected, allowing each sub-picture to include more than one rectangular slice, and the current picture includes only one rectangular slice; as well as If there is more than one slice in the current picture and the control syntax indicates that cross-slice boundary loop filtering is enabled, loop filtering is applied across slice boundaries.

Images