Improved sub-partition intra prediction

Abstract

Techniques for improving an intra-sub-partitioning (ISP) mode for partitioning a codec block into sub-blocks are described. In some cases, whether to enable the ISP mode for a codec block is based on a size constraint associated with a data unit (e.g., a VPDU, a transform block, etc.). For example, based on a size constraint associated with a VPDU, the ISP mode may be disabled for a codec block that crosses a VPDU boundary. In some cases, whether to enable the ISP mode may be based on a comparison of a width and / or height of the codec block with a size threshold corresponding to one or more maximum transform block sizes. In some cases, where the ISP mode is enabled for a codec block, a value for a flag defining a horizontal or vertical partitioning type for the codec block may be inferred based on the width and / or height of the codec block relative to one or more thresholds.

Description

Technical Field

[0001] The present application relates to video coding and decoding. More particularly, the present application relates to systems, methods and computer-readable media for performing improved video coding and decoding. Background Art

[0002] Many devices and systems allow video data to be processed and output for consumption. Digital video data includes large amounts of data to meet the needs of consumers and video providers. For example, consumers of video data expect the highest quality video with high fidelity, resolution, frame rate, etc. Therefore, the large amounts of video data required to meet these needs place a burden on the communication networks and devices that process and store video data.

[0003] Various video codec techniques can be used to compress video data. Video coding is performed according to one or more video codec standards. For example, video codec standards include High Efficiency Video Codec (HEVC), Advanced Video Codec (AVC), MPEG-2 Part 2 Codec (MPEG stands for Moving Picture Experts Group), VP9, ​​Alliance for Open Media (AOMedia) Video 1 (AV1), Basic Video Codec (EVC), etc. Video codecs typically utilize prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize redundancy present in video images or sequences. An important goal of video codec techniques is to compress video data into a form that uses a lower bit rate while avoiding or minimizing the reduction in video quality. As evolving video services become available, coding techniques with higher codec efficiency are needed. Summary of the invention

[0004] This article describes systems and methods for improving the use of intra-subpartitioning (ISP) mode for segmenting a codec block of video data into sub-blocks. In some examples, the decision on whether the ISP mode can be allowed for a codec block can be based on a size constraint. The size constraint can be implemented based on one or more size thresholds defined based on the size of the data unit. For example, the width size threshold can be defined as the width of the data unit, and the height size threshold can be defined as the height of the data unit. In some cases, a single threshold can be defined (e.g., when the width and height of the data unit are equal). The data unit may include a virtual pipeline data unit (VPDU), a transform block, or other data units or blocks. In an illustrative example, one or more size thresholds can be used to prevent the ISP mode from being enabled for a codec block that crosses a VPDU boundary. This constraint, known as a VPDU constraint, improves the processing efficiency of the VPDU by ensuring that the sub-blocks obtained from the segmented codec block are not processed separately in different VPDUs.

[0005] In some examples, whether to enable segmentation of a codec block using ISP mode may be determined based on a comparison of the width and height of the codec block to size thresholds corresponding to one or more maximum transform block sizes. In some examples, where ISP mode is enabled for a codec block, the optimal segmentation type (horizontal or vertical) may be determined based on the width and height of the codec block.

[0006] In some examples, a partitioning structure for ISP mode is provided. The partitioning structure ensures that the dimensions of the sub-blocks obtained by partitioning the codec block do not violate a size threshold such as a maximum transform block size. In some examples, a recursive tree structure is provided to ensure that the sub-partition split type is determined as horizontal or vertical in a manner that causes the size of the sub-block to meet the maximum transform block size threshold requirement. In some examples, the number of sub-blocks into which the codec block is partitioned is determined so that the dimensions of the sub-blocks meet the maximum transform block size threshold requirement. In some examples, enabling the ISP mode is adjusted to ensure that the dimensions of the sub-blocks obtained from the partitioned codec block meet the maximum transform block size threshold requirement.

[0007] According to at least one example, a device for encoding video data is provided. The device includes a memory and a processor implemented in a circuit. The processor is configured to and can obtain a current block of a picture of video data; determine whether the width of the current block of the picture is greater than a width size threshold and whether the height of the current block is greater than at least one of a height size threshold, the width size threshold corresponds to the width of the data unit and the height size threshold corresponds to the height of the data unit; based on whether the width of the current block is greater than the width size threshold and whether the height of the current block is greater than at least one of the height size threshold, determine whether to signal an intra-frame sub-partition mode flag for the current block, wherein the value of the intra-frame sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; and based on determining whether to signal the intra-frame sub-partition mode flag for the current block, generate a coded video bitstream including at least the current block.

[0008] In another example, a method for encoding video data is provided. The method includes obtaining a current block of a picture of the video data; determining whether a width of the current block of the picture is greater than a width size threshold and whether a height of the current block is greater than at least one of a height size threshold, the width size threshold corresponding to a width of a data unit and the height size threshold corresponding to a height of the data unit; determining whether to signal an intra-sub-partition mode flag for the current block based on whether the width of the current block is greater than the width size threshold and whether the height of the current block is greater than the height size threshold, wherein a value of the intra-sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; and generating an encoded video bitstream including at least the current block based on determining whether to signal the intra-sub-partition mode flag for the current block.

[0009] In another example, a non-transitory computer-readable medium is provided on which instructions are stored, which, when executed by one or more processors, cause the one or more processors to: obtain a current block of a picture of video data; determine whether a width of the current block of the picture is greater than a width size threshold and whether a height of the current block is greater than at least one of a height size threshold, wherein the width size threshold corresponds to a width of a data unit and the height size threshold corresponds to a height of the data unit; determine whether to signal an intra-sub-partition mode flag for the current block based on whether the width of the current block is greater than the width size threshold and whether the height of the current block is greater than the height size threshold, wherein a value of the intra-sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; and generate an encoded video bitstream including at least the current block based on determining whether the intra-sub-partition mode flag is to be signaled for the current block.

[0010] In another example, an apparatus for encoding video data is provided. The apparatus includes a component for obtaining a current block of a picture of the video data; a component for determining whether a width of the current block of the picture is greater than a width size threshold and whether a height of the current block is greater than at least one of a height size threshold, the width size threshold corresponding to a width of a data unit and the height size threshold corresponding to a height of the data unit; a component for determining whether to signal an intra-sub-partition mode flag for the current block based on whether the width of the current block is greater than the width size threshold and whether the height of the current block is greater than the height size threshold, wherein a value of the intra-sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; and a component for generating an encoded video bitstream including at least the current block based on determining whether to signal the intra-sub-partition mode flag for the current block.

[0011] In some aspects of the foregoing methods, apparatus, and computer-readable media, the width size threshold is equal to the height size threshold.

[0012] In some aspects of the above methods, apparatus, and computer-readable media, the width size threshold is different from the height size threshold.

[0013] In some aspects of the above methods, apparatuses, and computer-readable media, a value of the intra-subpartition mode flag equal to a first value specifies that the current block is divided into rectangular transform block subpartitions, and a value of the intra-subpartition mode flag equal to a second value specifies that the current block is not divided into rectangular transform block subpartitions.

[0014] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the width of the current block is greater than a width size threshold or the height of the current block is greater than a height size threshold; and based on the determination that the width of the current block is greater than the width size threshold or the height of the current block is greater than the height size threshold, determining that the value of the intra-frame sub-partition mode flag of the current block is equal to a second value.

[0015] Some aspects of the above methods, apparatuses, and computer-readable media also include determining that the width of the current block is less than or equal to a width size threshold and the height of the current block is less than or equal to a height size threshold; and based on the determination that the width of the current block is less than or equal to the width size threshold and the height of the current block is less than or equal to the height size threshold, determining to signal an intra-sub-partition mode flag of the current block. For example, the intra-sub-partition mode flag of the current block may be set equal to a first value or a second value.

[0016] Some aspects of the above methods, apparatuses, and computer-readable media also include determining that a width of the current block is less than or equal to a size threshold and a height of the current block is less than or equal to a size threshold, the size threshold corresponding to a maximum transform block size; and based on the determination that the width of the current block is less than or equal to the size threshold and the height of the current block is less than or equal to the size threshold, determining to signal an intra-sub-partition mode flag of the current block. For example, the intra-sub-partition mode flag of the current block may be set equal to a first value or a second value.

[0017] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the width of the current block is less than or equal to a size threshold or the height of the current block is less than or equal to the size threshold, and the size threshold corresponds to the maximum transform block size; and based on the determination that the width of the current block is less than or equal to the size threshold or the height of the current block is less than or equal to the size threshold, determining that the value of the intra-frame sub-partition mode flag of the current block is a second value.

[0018] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the width of the current block is less than or equal to a width size threshold and the height of the current block is less than or equal to a height size threshold; and based on the determination that the width of the current block is less than or equal to the width size threshold and the height of the current block is less than or equal to the height size threshold, determining that the value of the intra-frame sub-partition mode flag of the current block is equal to the first value.

[0019] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the width of the current block is less than or equal to a size threshold and the height of the current block is less than or equal to the size threshold, and the size threshold corresponds to the maximum transform block size; and based on the determination that the width of the current block is less than or equal to the size threshold and the height of the current block is less than or equal to the size threshold, determining that the value of the intra-frame sub-partition mode flag of the current block is equal to the first value.

[0020] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the width of the current block is less than or equal to a size threshold or the height of the current block is less than or equal to the size threshold, and the size threshold corresponds to the maximum transform block size; and based on the determination that the width of the current block is less than or equal to the size threshold or the height of the current block is less than or equal to the size threshold, determining that the value of the intra-frame sub-partition mode flag of the current block is a second value.

[0021] Some aspects of the above-mentioned methods, apparatuses and computer-readable media also include determining that an intra-frame sub-partition mode flag is to be signaled for the current block; and based on the determination that the intra-frame sub-partition mode flag is to be signaled for the current block, including the intra-frame sub-partition mode flag of the current block in the encoded video bitstream.

[0022] Some aspects of the above-mentioned methods, apparatuses and computer-readable media also include determining that at least one of the width and height of the current block is greater than a size threshold corresponding to the maximum transform block size; and based on the determination that the width or height of the current block is greater than the size threshold corresponding to the maximum transform block size, determining a value of an intra-frame sub-partition split flag of the current block, the intra-frame sub-partition split flag specifying whether a split type used to divide the current block is horizontal or vertical.

[0023] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the width of the current block is greater than a size threshold; and based on the determination that the width of the current block is greater than the size threshold, determining a first segmentation value of the intra-frame sub-partition segmentation flag, wherein the first segmentation value corresponds to a vertical segmentation type.

[0024] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the height of the current block is greater than a size threshold; and based on the determination that the height of the current block is greater than the size threshold, determining a second segmentation value of the intra-frame sub-partition segmentation flag, wherein the second segmentation value corresponds to a horizontal segmentation type.

[0025] Some aspects of the above methods, apparatuses, and computer-readable media further include dividing the current block vertically or horizontally into sub-partitions based on a value of an intra-sub-partition split flag of the current block.

[0026] In some aspects of the foregoing methods, apparatus, and computer-readable media, the current block is an intra-predicted block.

[0027] In some aspects of the foregoing methods, apparatus, and computer-readable media, the data unit is a virtual pipeline data unit (VPDU).

[0028] In some aspects of the above methods, apparatus, and computer-readable media, the data unit is a transform block. In some cases, the width size threshold and the height size threshold are equal to a maximum transform block size.

[0029] In another example, a method for decoding video data is provided. The method includes obtaining an encoded video bitstream including video data; determining that an intra-sub-partition split flag for a current block of the video data is not present in the encoded video bitstream, the intra-sub-partition split flag specifying whether a split type of an intra-sub-partition mode for the current block is horizontal or vertical; determining that at least one of a width and a height of the current block is greater than a size threshold corresponding to a maximum transform block size; and determining a value of the intra-sub-partition split flag of the current block based on whether the width or the height of the current block is greater than the size threshold.

[0030] In another example, a device for decoding video data is provided. The device includes a memory and a processor implemented in a circuit. The processor is configured to and can obtain an encoded video bitstream including video data; determine that there is no intra-frame sub-partition split flag for a current block of video data in the encoded video bitstream, the intra-frame sub-partition split flag specifying whether the split type of the intra-frame sub-partition mode for the current block is horizontal or vertical; determine that at least one of the width and height of the current block is greater than a size threshold corresponding to the maximum transform block size; and determine the value of the intra-frame sub-partition split flag of the current block based on whether the width or height of the current block is greater than the size threshold.

[0031] In another example, a non-transitory computer-readable medium is provided on which instructions are stored, which, when executed by one or more processors, cause the one or more processors to: obtain an encoded video bitstream including video data; determine that there is no intra-frame sub-partition split flag for a current block of video data in the encoded video bitstream, the intra-frame sub-partition split flag specifying whether the split type of the intra-frame sub-partition mode for the current block is horizontal or vertical; determine that at least one of the width and height of the current block is greater than a size threshold corresponding to a maximum transform block size; and determine a value of the intra-frame sub-partition split flag of the current block based on whether the width or height of the current block is greater than the size threshold.

[0032] In another example, an apparatus for decoding video data is provided. The apparatus includes a component for obtaining an encoded video bitstream including the video data; a component for determining that an intra-sub-partition split flag for a current block of the video data is not present in the encoded video bitstream, the intra-sub-partition split flag specifying whether a split type of an intra-sub-partition mode for the current block is horizontal or vertical; a component for determining that at least one of a width and a height of the current block is greater than a size threshold corresponding to a maximum transform block size; and a component for determining a value of the intra-sub-partition split flag of the current block based on whether the width or the height of the current block is greater than the size threshold.

[0033] Some aspects of the above-mentioned method, apparatus and computer-readable medium also include determining that an intra-frame sub-partition mode flag is enabled for the current block; and based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, determining that there is no intra-frame sub-partition split flag in the encoded video bitstream, wherein based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, the current block is divided into rectangular transform block sub-partitions using the intra-frame sub-partition mode.

[0034] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the width of the current block is greater than a size threshold; and based on the determination that the width of the current block is greater than the size threshold, determining a first segmentation value of the intra-frame sub-partition segmentation flag, wherein the first segmentation value corresponds to a vertical segmentation type.

[0035] Some aspects of the above methods, apparatus, and computer-readable media also include using a vertical partitioning type to partition the current block into two or more sub-blocks, wherein based on the vertical partitioning type, a corresponding width of each of the two or more sub-blocks is smaller than a width of the current block.

[0036] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that the height of the current block is greater than a size threshold; and based on the determination that the height of the current block is greater than the size threshold, determining a second segmentation value of the intra-frame sub-partition segmentation flag, wherein the second segmentation value corresponds to a horizontal segmentation type.

[0037] Some aspects of the above-mentioned methods, apparatuses and computer-readable media also include using a horizontal partitioning type to partition the current block into two or more sub-blocks, wherein based on the horizontal partitioning type, a corresponding height of each of the two or more sub-blocks is less than a height of the current block.

[0038] Some aspects of the above-mentioned methods, devices and computer-readable media also include determining that there is no intra-frame sub-partition mode flag in the encoded video bitstream for enabling the intra-frame sub-partition mode for the second block of video data; and determining the mode value of the intra-frame sub-partition mode flag for the second block based on the absence of the intra-frame sub-partition mode flag for the second block in the encoded video bitstream.

[0039] In some aspects of the above-mentioned methods, devices and computer-readable media, the absence of an intra-frame sub-partition mode flag for the second block in the encoded video bitstream indicates that the width of the second block is greater than a width size threshold or the height of the second block is greater than at least one of a height size threshold, the width size threshold corresponds to the width of the data unit, and the height size threshold corresponds to the height of the data unit.

[0040] In some aspects of the foregoing methods, apparatus, and computer-readable media, the width size threshold is equal to the height size threshold.

[0041] In some aspects of the foregoing methods, apparatus, and computer-readable media, the width size threshold is not equal to the height size threshold.

[0042] In some aspects of the foregoing methods, apparatus, and computer-readable media, the data unit is a virtual pipeline data unit (VPDU).

[0043] In some aspects of the above methods, apparatus, and computer-readable media, the data unit is a transform block. In some cases, the width size threshold and the height size threshold are equal to a maximum transform block size.

[0044] In some aspects of the foregoing methods, apparatus, and computer-readable media, the current block is an intra-predicted block.

[0045] Some aspects of the above-mentioned methods, apparatuses, and computer-readable media also include performing block-based partitioning of a block of video data to generate one or more codec blocks including a current block; determining to enable an intra-frame sub-partitioning mode for the current block; and based on determining to enable the intra-frame sub-partitioning mode for the current block, using the intra-frame sub-partitioning mode to partition the current block into two or more sub-blocks.

[0046] Some aspects of the above methods, apparatus, and computer-readable media also include reconstructing the two or more sub-blocks by applying the corresponding two or more residual values ​​to the two or more sub-blocks.

[0047] In some aspects of the above methods, apparatus, and computer-readable media, the apparatus includes a mobile device having a camera for capturing one or more pictures.

[0048] Some aspects of the above methods, apparatus, and computer-readable media also include a display for displaying one or more pictures.

[0049] This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used alone to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this patent, any or all drawings, and each claim.

[0050] The above and other features and embodiments will become more fully apparent upon reference to the following description, claims and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Exemplary embodiments of the present application are described in detail below with reference to the following drawings:

[0052] Figure 1 is a block diagram illustrating examples of encoding devices and decoding devices according to some examples;

[0053] Figure 2is a conceptual diagram showing an example of partitioning of blocks according to some examples;

[0054] Figure 3 is a conceptual diagram showing another example of division of blocks according to some examples;

[0055] Figure 4 is a conceptual diagram illustrating examples of codec blocks according to some examples;

[0056] Figure 5 is a conceptual diagram illustrating an example of a VPDU containing four blocks according to some examples;

[0057] Figure 6 is a conceptual diagram illustrating an example of a current block spanning multiple VPDUs according to some examples;

[0058] Figure 7 is a flow chart illustrating an example of a process for processing video data according to some embodiments;

[0059] Figure 8 is a flow chart illustrating an example of a process for processing video data according to some embodiments;

[0060] Fig. 9 is a flow chart illustrating an example of a process for processing video data according to some embodiments;

[0061] Fig.10 is a block diagram illustrating an example video encoding device according to some examples; and

[0062] Fig.11 is a block diagram illustrating an example video decoding device according to some examples. DETAILED DESCRIPTION

[0063] Some aspects and embodiments of the present disclosure are provided below. Some of these aspects and embodiments can be applied independently, and some of them can be applied in combination, which will be apparent to those skilled in the art. In the following description, for the purpose of explanation, specific details are set forth to provide a thorough understanding of the embodiments of the present application. However, it is apparent that various embodiments can be practiced without these specific details. The accompanying drawings and description are not intended to be limiting.

[0064] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. On the contrary, the following description of the exemplary embodiments will provide an enabling description for implementing the exemplary embodiments for those skilled in the art. It should be understood that various changes may be made to the functions and arrangements of the elements without departing from the spirit and scope of the present application as set forth in the appended claims.

[0065] Video codecs implement video compression techniques to efficiently encode and decode video data. Video compression techniques may include applying different prediction modes, including spatial prediction (e.g., prediction within a frame or intra-frame prediction), temporal prediction (e.g., prediction between frames or inter-frame prediction), inter-layer prediction (across different layers of video data), and / or other prediction techniques to reduce or remove redundancy inherent in video sequences. A video encoder may divide each picture of an original video sequence into rectangular areas called video blocks or codec units (described in more detail below). These video blocks may be encoded using specific prediction modes.

[0066] The video may be divided into one or more groups of smaller blocks in one or more ways. Blocks may include codec tree blocks, prediction blocks, transform blocks, or other suitable blocks. Unless otherwise specified, references to "blocks" in general may refer to such video blocks (e.g., codec tree blocks, codec blocks, prediction blocks, transform blocks, or other suitable blocks or sub-blocks, as would be understood by a person of ordinary skill). In addition, each of these blocks may also be referred to interchangeably herein as a "unit" (e.g., codec tree unit (CTU), codec unit, prediction unit (PU), transform unit (TU), etc.). In some cases, a unit may indicate a codec logic unit encoded in a bitstream, while a block may indicate a portion of a video frame buffer for which a process is directed.

[0067] For inter prediction mode, the video encoder can search for a block similar to the block being encoded in a frame (or picture) located at another time position, which is called a reference frame or reference picture. The video encoder can limit the search to a certain spatial displacement from the block to be encoded. A two-dimensional (2D) motion vector including a horizontal displacement component and a vertical displacement component can be used to locate the best match. For intra prediction mode, the video encoder can use spatial domain prediction techniques to form a prediction block based on data from previously encoded neighboring blocks in the same picture.

[0068] The video encoder can determine the prediction error. For example, the prediction can be determined as the difference between the pixel values ​​in the block being encoded and the predicted block. The prediction error can also be referred to as a residual. The video encoder can also apply a transform (e.g., a discrete cosine transform (DCT) or other suitable transform) to the prediction error to generate transform coefficients. After the transform, the video encoder can quantize the transform coefficients. The quantized transform coefficients and motion vectors can be represented using syntax elements and, together with the control information, form a coded representation of the video sequence. In some cases, the video encoder can entropy encode and decode the syntax elements to further reduce the number of bits required for their representation.

[0069] The video decoder can use the syntax elements and control information discussed above to construct predictive data (e.g., a predictive block) for decoding the current frame. For example, the video decoder can add the prediction block and the compressed prediction error. The video decoder can determine the compressed prediction error by weighting the transform basis function using quantized coefficients. The difference between the reconstructed frame and the original frame is called the reconstructed error.

[0070] Described herein are systems and methods for improving the use of intra-frame sub-division or intra-frame sub-division (ISP) mode for segmenting a codec block of video data into sub-blocks according to one or more restrictions and / or constraints. In some examples, the decision whether to perform ISP mode on a codec block is based on a size constraint. The size constraint can be based on one or more size thresholds defined using the size of a data unit. For example, as described in more detail below, a width size threshold can be defined as the width of a data unit, and a height size threshold can be defined as the height of a data unit. In some cases, a single threshold can be defined (e.g., when the width and height of a data unit are equal). A data unit can include a virtual pipeline data unit (VPDU), a transform block, or other data units or blocks. In an illustrative example, a VPDU constraint is defined herein that prevents ISP mode from being enabled for a codec block that crosses a virtual pipeline data unit (VPDU) boundary. This VPDU constraint can improve the processing efficiency of the VPDU by ensuring that sub-blocks obtained from segmenting a codec block using ISP are not processed separately in different VPDUs. In some examples, the VPDU constraint may be applied by the encoder, wherein an ISP mode flag is not signaled for the codec block in or with the coded video bitstream when the codec block violates the VPDU constraint. In such an example, based on not signaling the ISP mode flag in or with the coded video bitstream, the decoder may infer that the value of the ISP mode flag is a value indicating that ISP mode is not enabled for the codec block (e.g., a value of 0).

[0071] A split flag (in some cases, referred to herein as an intra-sub-partition split flag) can be signaled in the encoded video bitstream and used to determine the type of ISP split (e.g., horizontal type or vertical type) to be performed on a codec block for which the ISP mode is enabled. In an example aspect, an improved technique for determining and signaling a split flag is provided. For example, an optimal ISP split type can be determined for a codec block based on a specific dimension of the codec block relative to a maximum transform block size, which can avoid undesirable split types that may result according to existing implementations. In one example, when the ISP mode is enabled for a codec block, the determination of the type of ISP split (e.g., horizontal type or vertical type) to be performed on the codec block can be determined based on a comparison of the width and / or height of the codec block with a size threshold corresponding to the maximum transform block size. For example, the value of the split flag of the codec block can be determined based on a comparison of the width and / or height of the codec block with a maximum transform block size threshold, and the value of the split flag can be used to determine the ISP split type (e.g., horizontal or vertical).

[0072] In some examples, a partitioning structure of the ISP mode may be provided to ensure that the dimensions of the sub-blocks obtained by partitioning the codec block do not violate a size threshold such as a maximum transform block size. In some examples, a recursive tree structure is provided to ensure that the sub-partition split type is determined as horizontal or vertical in a manner that causes the size of the sub-block to meet the maximum transform block size threshold requirement. In some examples, the number of sub-blocks into which the codec block is partitioned is determined so that the dimensions of the sub-blocks meet the maximum transform block size threshold requirement. In some examples, enabling the ISP mode is adjusted to ensure that the dimensions of the sub-blocks obtained from the partitioned codec block meet the maximum transform block size threshold requirement.

[0073] Figure 11 is a block diagram showing an example of a system 100 including an encoding device 104 and a decoding device 112. The encoding device 104 may be part of a source device, and the decoding device 112 may be part of a receiving device. The source device and / or the receiving device may include an electronic device, such as a mobile or fixed telephone handset (e.g., a smart phone, a cellular phone, etc.), a desktop computer, a laptop or notebook computer, a tablet computer, a set-top box, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, an Internet Protocol (IP) camera, or any other suitable electronic device. In some examples, the source device and the receiving device may include one or more wireless transceivers for wireless communication. The encoding techniques described herein are applicable to video encoding and decoding in various multimedia applications, including streaming video transmission (e.g., via the Internet), television broadcasting or transmission, encoding of digital video stored on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the system 100 may support one-way or two-way video transmission to support applications such as video conferencing, video streaming, video playback, video broadcasting, gaming, and / or video telephony.

[0074] The encoding device 104 (or encoder) can be used to encode video data using a video codec standard or protocol to generate an encoded video bitstream. Examples of video codec standards include ITU-T H.261, ISO / IEC MPEG-1 Vision, ITU-T H.262 or ISO / IEC MPEG-2 Vision, ITU-T H.263, ISO / IEC MPEG-4 Vision, ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), including its scalable video codec (SVC) and multi-view video codec (MVC) extensions, and high-efficiency video codec (HEVC) or ITU-T H.265. There are various extensions of HEVC for multi-layer video codecs, including range and screen content codec extensions, 3D video codec (3D-HEVC) and multi-view extensions (MV-HEVC) and scalable extensions (SHVC). HEVC and its extensions were developed by the Joint Collaboration Team on Video Coding and Decoding (JCT-VC) of the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG), and the Joint Collaboration Team on 3D Video Codec Extensions (JCT-3V).

[0075] MPEG and ITU-T VCEG have also formed the Joint Video Exploration Team (JVET) to explore new codec tools for the next generation of video codec standards, named Versatile Video Codec (VVC). The reference software is called the VVC Test Model (VTM). The goal of VVC is to provide significant improvements in compression performance over the existing HEVC standard, facilitating the deployment of higher quality video services and emerging applications (e.g., 360° immersive multimedia, high dynamic range (HDR) video, etc.). VP9, ​​Alliance for Open Media (AOMedia) Video 1 (AV1), and Essential Video Codec (EVC) are other video codec standards to which the techniques described in this article can be applied.

[0076] Many of the embodiments described herein may be performed using video codecs such as VTM, VVC, HEVC, AVC, and / or their extensions. However, the techniques and systems described herein may also be applicable to other codec standards, such as MPEG, JPEG (or other codec standards for still images), VP9, ​​AV1, their extensions, or other suitable codec standards that are already available or not yet available or developed. Therefore, although the techniques and systems described herein may be described with reference to a particular video codec standard, a person of ordinary skill in the art will understand that the description should not be interpreted as being applicable only to that particular standard.

[0077] Reference Figure 1 , video source 102 may provide video data to encoding device 104. Video source 102 may be part of a source device, or may be part of a device other than a source device. Video source 102 may include a video capture device (e.g., a video camera, a camera phone, a video phone, etc.), a video archive containing stored video, a video server or content provider that provides video data, a video feed interface that receives video from a video server or content provider, a computer graphics system for generating computer graphics video data, a combination of these sources, or any other suitable video source.

[0078] The video data from video source 102 may include one or more input pictures or frames. A picture or frame is a still image that is, in some cases, part of a video. In some examples, the data from video source 102 may be a still image that is not part of a video. In HEVC, VVC, and other video codec specifications, a video sequence may include a series of pictures. A picture may include three sample arrays, denoted S L , S Cb and S Cr . S L is a two-dimensional array of brightness samples, S Cb is a two-dimensional array of Cb chromaticity samples, S Cris a two-dimensional array of Cr chrominance samples. Chrominance samples may also be referred to herein as "chrominance" samples. In other examples, a picture may be monochrome and may include only an array of luma samples.

[0079] The encoder engine 106 (or encoder) of the encoding device 104 encodes the video data to generate a coded video bitstream. In some examples, the coded video bitstream (or "video bitstream" or "bitstream") is a series of one or more coded video sequences. The coded video sequence (CVS) includes a series of access units (AUs) starting from an AU having a random access point picture in the base layer and having certain attributes, until and not including the next AU having a random access point picture in the base layer and having certain attributes. For example, certain attributes of the random access point picture that starts the CVS may include a RASL flag (e.g., NoRaslOutputFlag) equal to 1. Otherwise, the random access point picture (with a RASL flag equal to 0) does not start the CVS. The access unit (AU) includes one or more coded pictures and control information corresponding to the coded pictures sharing the same output time. The coded slices of the picture are encapsulated into a data unit called a network abstraction layer (NAL) unit at the bitstream level. For example, a HEVC video bitstream may include one or more CVSs including NAL units. Each NAL unit has a NAL unit header. In one example, the header is one byte for H.264 / AVC (except for multi-layer extensions) and two bytes for HEVC. Syntax elements in the NAL unit header use designated bits and are therefore visible to all types of systems and transport layers (e.g., transport streams, real-time transport (RTP) protocols, file formats, etc.).

[0080] There are two types of NAL units in the HEVC standard, including video codec layer (VCL) NAL units and non-VCL NAL units. The VCL NAL unit includes a slice or slice segment of coded picture data (described below), and the non-VCL NAL unit includes control information related to one or more coded pictures. In some cases, the NAL unit can be referred to as a packet. The HEVC AU includes a VCL NAL unit containing coded picture data and a non-VCL NAL unit corresponding to the coded picture data (if any).

[0081] The NAL unit may contain a sequence of bits that form a coded representation of video data (e.g., a coded video bitstream, a CVS of a bitstream, etc.), such as a coded representation of a picture in a video. The encoder engine 106 generates a coded representation of a picture by dividing each picture into a plurality of slices. A slice is independent of other slices so that information in a slice is coded or decoded without relying on data from other slices within the same picture. A slice includes one or more slice segments, and a slice segment includes an independent slice segment and one or more dependent slice segments that depend on previous slice segments (if present).

[0082] In HEVC, slices are then divided into codec tree blocks (CTBs) of luma samples and chroma samples. A CTB for luma samples and one or more CTBs for chroma samples, together with the syntax of the samples, are called a codec tree unit (CTU). A CTU may also be called a "tree block" or a "largest codec unit" (LCU). A CTU is the basic processing unit for HEVC encoding. A CTU can be split into multiple codec units (CUs) of different sizes. A CU contains an array of luma and chroma samples called a codec block (CB).

[0083] Luma and chroma CBs can be further divided into prediction blocks (PBs). A PB is a block of samples of a luminance component or a chroma component that uses the same motion parameters for inter-frame prediction or intra-frame block copy (IBC) prediction (when available or enabled for use). A luminance PB and one or more chroma PBs, together with associated syntax, form a prediction unit (PU). For inter-frame prediction, a set of motion parameters (e.g., one or more motion vectors, reference indices, etc.) is signaled for each PU in the bitstream and used for inter-frame prediction of the luminance PB and one or more chroma PBs. Motion parameters may also be referred to as motion information. A CB may also be divided into one or more transform blocks (TBs). A TB represents a square block of samples of a color component on which a residual transform (e.g., the same two-dimensional transform in some cases) is applied to encode and decode a prediction residual signal. A transform unit (TU) represents a TB of luminance and chroma samples and corresponding syntax elements. Transform coding and decoding is described in more detail below.

[0084] The size of the CU corresponds to the size of the codec mode and can be square in shape. For example, the size of the CU can be 8×8 samples, 16×16 samples, 32×32 samples, 64×64 samples, or any other suitable size that reaches the size of the corresponding CTU. The phrase "N×N" is used herein to refer to the pixel dimensions of the video block in terms of vertical and horizontal dimensions (e.g., 8 pixels × 8 pixels). The pixels in the block can be arranged in rows and columns. In some embodiments, the block may not have the same number of pixels in the horizontal direction as in the vertical direction. For example, the syntax data associated with the CU may describe the division of the CU into one or more PUs. The division mode may be different between whether the CU is encoded in an intra-frame prediction mode or an inter-frame prediction mode. The PU can be divided into a non-square shape. For example, the syntax data associated with the CU can also describe the division of the CU into one or more TUs according to the CTU. The TU can be square or non-square in shape.

[0085] According to the HEVC standard, a transform unit (TU) may be used to perform a transform. The TU may vary for different CUs. The size of the TU may be determined based on the size of the PU within a given CU. The TU may be the same size as or smaller than the PU. In some examples, a quadtree structure called a residual quadtree (RQT) may be used to subdivide the residual samples corresponding to the CU into smaller units. The leaf nodes of the RQT may correspond to the TU. The pixel difference values ​​associated with the TU may be transformed to generate transform coefficients. The transform coefficients may then be quantized by the encoder engine 106.

[0086] Once the picture of the video data is divided into CUs, the encoder engine 106 predicts each PU using a prediction mode. The prediction unit or prediction block is then subtracted from the original video data to obtain a residual (as described below). For each CU, the prediction mode can be signaled in the bitstream using syntax data. The prediction mode can include intra-frame prediction (or intra-picture prediction) or inter-frame prediction (or inter-picture prediction). Intra-frame prediction exploits the correlation between spatially adjacent samples in a picture. For example, using intra-frame prediction, each PU is predicted from adjacent image data in the same picture, for example, using DC prediction to find the average value of the PU, using plane prediction to fit a plane surface to the PU, using directional prediction to extrapolate from adjacent data, or using any other suitable type of prediction. Inter-frame prediction uses temporal correlation between pictures to derive motion-compensated predictions for blocks of image samples. For example, using inter-frame prediction, each PU is predicted using motion-compensated predictions of image data from one or more reference pictures (before or after the current picture in output order). For example, a decision can be made at the CU level whether to use inter-picture prediction or intra-picture prediction to encode or decode a picture area.

[0087] The encoder engine 106 and the decoder engine 116 (described in more detail below) can be configured to operate according to VVC. According to VVC, the video codec (e.g., the encoder engine 106 and / or the decoder engine 116) divides the picture into multiple codec tree units (CTUs) (wherein the CTB of the luma sample and one or more CTBs of the chroma sample, together with the syntax elements of the sample, are referred to as CTUs). The video codec can divide the CTU according to a tree structure (e.g., a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure removes the concept of multiple partition types, such as the separation of CU, PU, ​​and TU of HEVC. The QTBT structure includes two levels: a first level divided according to quadtree partitioning and a second level divided according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to the codec units (CUs).

[0088] In the MTT partition structure, blocks can be partitioned using quadtree partitioning, binary tree partitioning, and one or more types of ternary tree partitioning. Ternary tree partitioning is a partitioning in which a block is divided into three sub-blocks. In some examples, ternary tree partitioning divides a block into three sub-blocks without dividing the original block through the center. The partitioning types in MTT (e.g., quadtree, binary tree, and ternary tree) can be symmetric or asymmetric.

[0089] In some examples, the video codec may use a single QTBT or MTT structure to represent each of the luma and chroma components, while in other examples, the video codec may use two or more QTBT or MTT structures, such as one QTBT or MTT structure for the luma component and another QTBT or MTT structure for two chroma components (or two QTBT and / or MTT structures for each chroma component).

[0090] The video codec may be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, or other partitioning structures in accordance with HEVC. For purposes of illustration, the description herein may refer to QTBT partitioning. However, it should be understood that the techniques of the present disclosure may also be applied to video codecs configured to use quadtree partitioning or other types of partitioning.

[0091] In some examples, a slice type is assigned to one or more slices of a picture. Slice types include I slices, P slices, and B slices. I slices (intra, independently decodable) are slices of a picture that are encoded and decoded only by intra prediction, and are therefore independently decodable because an I slice only requires data within the frame to predict any prediction unit or prediction block of the slice. P slices (unidirectionally predicted frames) are slices of a picture that can be encoded and decoded using intra prediction and unidirectional inter prediction. Each prediction unit or prediction block within a P slice is encoded and decoded using intra prediction or inter prediction. When inter prediction is applied, the prediction unit or prediction block is predicted by only one reference picture, and therefore the reference samples come from only one reference region of a frame. B slices (bidirectionally predictive frames) are slices of a picture that can be encoded and decoded using intra prediction and inter prediction (e.g., bidirectional prediction or unidirectional prediction). The prediction unit or prediction block of a B slice may be bidirectionally predicted from two reference pictures, where each picture contributes one reference region and the sample sets of the two reference regions are weighted (e.g., with equal weights or with different weights) to generate a prediction signal for the bidirectionally predicted block. As described above, the slices of a picture are independently coded. In some cases, a picture may be coded as only one slice.

[0092] As described above, intra-picture prediction exploits the correlation between spatially adjacent samples within a picture. There are multiple intra-frame prediction modes (also referred to as "intra-frame modes"). In some examples, intra-frame prediction of luma blocks includes 35 modes, including a plane mode, a DC mode, and 33 angle modes (e.g., a diagonal intra-frame prediction mode and an angle mode adjacent to the diagonal intra-frame prediction mode). The indexes of the 35 modes of intra-frame prediction are shown in Table 1 below. In other examples, more intra-frame modes may be defined, including prediction angles that may not yet be represented by the 33 angle modes. In other examples, the prediction angles associated with the angle modes may be different from those used in HEVC.

[0093] Intra prediction mode Related Name 0 INTRA_PLANAR 1 INTRA_DC 2..34 INTRA_ANGULAR2..INTRA_ANGULAR34

[0094] Table 1 – Specification of intra prediction modes and associated names

[0095] Inter-picture prediction uses temporal correlation between pictures to derive motion compensated prediction for blocks of image samples. Using a translational motion model, the position of a block in a previously decoded picture (reference picture) is indicated by a motion vector (Δx, Δy), where Δx specifies the horizontal displacement of the reference block and Δy specifies the vertical displacement of the reference block relative to the position of the current block. In some cases, the motion vector (Δx, Δy) may be integer sample precision (also referred to as integer precision), in which case the motion vector points to an integer pixel grid (or integer pixel sampling grid) of a reference frame. In some cases, the motion vector (Δx, Δy) may be fractional sample precision (also referred to as fractional pixel precision or non-integer precision) to more accurately capture the motion of potential objects without being limited to the integer pixel grid of a reference frame. The precision of the motion vector may be represented by the quantization level of the motion vector. For example, the quantization level may be integer precision (e.g., 1 pixel) or fractional pixel precision (e.g., 1 / 4 pixel, 1 / 2 pixel, or other sub-pixel value). When the corresponding motion vector has fractional sample precision, interpolation is applied to the reference picture to derive a prediction signal. For example, the samples available at integer positions may be filtered (e.g., using one or more interpolation filters) to estimate values ​​at fractional positions. A previously decoded reference picture is indicated by a reference index (refIdx) of a reference picture list. Motion vectors and reference indices may be referred to as motion parameters. Two types of inter-picture prediction may be performed, including unidirectional prediction and bidirectional prediction.

[0096] In inter-frame prediction using bi-prediction, two sets of motion parameters (Δx 0 ,y 0 ,refIdx 0 and Δx 1 ,y 1 ,refIdx 1 ) is used to generate two motion compensated predictions (from the same reference picture or possibly from different reference pictures). For example, for bidirectional prediction, two motion compensated prediction signals are used for each prediction block, and B prediction units are generated. The two motion compensated predictions are then combined to obtain the final motion compensated prediction. For example, the two motion compensated predictions can be combined by averaging. In another example, weighted prediction can be used, in which case different weights can be applied to each motion compensated prediction. The reference pictures that can be used for bidirectional prediction are stored in two separate lists (denoted as List 0 and List 1). The motion parameters can be derived at the encoder using a motion estimation process.

[0097] In inter-frame prediction using unidirectional prediction, a set of motion parameters (Δx 0 ,y 0 ,refIdx 0) is used to generate motion compensated prediction from a reference picture. For example, for unidirectional prediction, each prediction block uses at most one motion compensated prediction signal and generates P prediction units.

[0098] The PU may include data related to the prediction process (e.g., motion parameters or other suitable data). For example, when the PU is encoded using intra prediction, the PU may include data describing the intra prediction mode of the PU. As another example, when the PU is encoded using inter prediction, the PU may include data defining the motion vector of the PU. The data defining the motion vector of the PU may describe, for example, the horizontal component of the motion vector (Δx), the vertical component of the motion vector (Δy), the resolution of the motion vector (e.g., integer precision, quarter-pixel precision, or eighth-pixel precision), the reference picture to which the motion vector points, a reference index, a reference picture list of the motion vector (e.g., list 0, list 1, or list C), or any combination thereof.

[0099] After performing prediction using intra-frame and / or inter-frame prediction, the encoding device 104 may perform transformation and quantization. For example, after prediction, the encoder engine 106 may calculate a residual value corresponding to the PU. The residual value may include a pixel difference value between a current pixel block (PU) being encoded and decoded and a prediction block (e.g., a predicted version of the current block) used to predict the current block. For example, after generating a prediction block (e.g., issuing an inter-frame prediction or an intra-frame prediction), the encoder engine 106 may generate a residual block by subtracting a prediction block generated by a prediction unit from the current block. The residual block includes a set of pixel difference values ​​that quantize the difference between the pixel values ​​of the current block and the pixel values ​​of the prediction block. In some examples, the residual block may be represented in a two-dimensional block format (e.g., a two-dimensional matrix or array of pixel values). In such an example, the residual block is a two-dimensional representation of pixel values.

[0100] Any residual data that may remain after prediction is performed is transformed using a block transform, which may be based on a discrete cosine transform, a discrete sine transform, an integer transform, a wavelet transform, other suitable transform functions, or any combination thereof. In some cases, one or more block transforms (e.g., of size 32×32, 16×16, 8×8, 4×4, or other suitable sizes) may be applied to the residual data in each CU. In some embodiments, a TU may be used for a transform and quantization process implemented by the encoder engine 106. A given CU with one or more PUs may also include one or more TUs. As described in further detail below, a block transform may be used to transform residual values ​​into transform coefficients, which may then be quantized and scanned using TUs to produce serialized transform coefficients for entropy encoding and decoding.

[0101] In some embodiments, after performing intra-predictive or inter-predictive coding and decoding using the PU of the CU, the encoder engine 106 can calculate the residual data of the TU of the CU. The PU may include pixel data in the spatial domain (or pixel domain). After applying the block transform, the TU may include coefficients in the transform domain. As described above, the residual data may correspond to the pixel difference between the pixel of the uncoded picture and the predicted value corresponding to the PU. The encoder engine 106 may form a TU including the residual data of the CU, and then the TU may be transformed to generate the transform coefficients of the CU.

[0102] The encoder engine 106 can perform quantization of the transform coefficients. Quantization provides further compression by quantizing the transform coefficients to reduce the amount of data used to represent the coefficients. For example, quantization can reduce the bit depth associated with some or all coefficients. In one example, a coefficient with an n-bit value can be rounded down to an m-bit value during quantization, where n is greater than m.

[0103] Once quantization is performed, the coded video bitstream includes quantized transform coefficients, prediction information (e.g., prediction mode, motion vectors, block vectors, etc.), partition information, and any other suitable data, such as other syntax data. The different elements of the coded video bitstream can then be entropy encoded by the encoder engine 106. In some examples, the encoder engine 106 may scan the quantized transform coefficients using a predefined scan order to produce a serialized vector that can be entropy encoded. In some examples, the encoder engine 106 may perform adaptive scanning. After scanning the quantized transform coefficients to form a vector (e.g., a one-dimensional vector), the encoder engine 106 may entropy encode the vector. For example, the encoder engine 106 may use context-adaptive variable length coding, context-adaptive binary arithmetic coding, grammar-based context-adaptive binary arithmetic coding, probability interval partitioning entropy coding, or another suitable entropy coding technique.

[0104] As mentioned above, the HEVC bitstream includes a set of NAL units, including VCL NAL units and non-VCL NAL units. The VCL NAL unit includes coded and decoded picture data that forms a coded and decoded video bitstream. For example, there is a bit sequence that forms a coded and decoded video bitstream in the VCL NAL unit. In addition to other information, the non-VCL NAL unit may also contain a parameter set with high-level information related to the coded and decoded video bitstream. For example, a parameter set may include a video parameter set (VPS), a sequence parameter set (SPS), and a picture parameter set (PPS). Examples of the goals of the parameter set include bit rate efficiency, error resistance, and providing a system layer interface. Each slice references a single active PPS, SPS, and VPS to access information that the decoding device 112 can use to decode the slice. An identifier (ID) can be encoded and decoded for each parameter set, including a VPS ID, an SPS ID, and a PPS ID. The SPS includes an SPS ID and a VPS ID. The PPS includes a PPS ID and an SPS ID. Each slice header includes a PPS ID. Using the ID, an active parameter set can be identified for a given slice.

[0105] The PPS includes information applied to all slices in a given picture. Therefore, all slices in a picture refer to the same PPS. Slices in different pictures may also refer to the same PPS. The SPS includes information applied to all pictures in the same coded video sequence (CVS) or bitstream. As previously described, a coded video sequence is a series of access units (AUs) that start with a random access point picture in the base layer (e.g., an instantaneous decoding reference (IDR) picture or a broken link access (BLA) picture or other appropriate random access point picture) and have certain properties (as described above) until and not including the next AU with a random access point picture in the base layer and certain properties (or the end of the bitstream). Within a coded video sequence, the information in the SPS may not change with changes in the picture. The pictures in a coded video sequence may use the same SPS. The VPS includes information applied to all layers within a coded video sequence or bitstream. The VPS includes a syntax structure having syntax elements applied to the entire coded video sequence. In some embodiments, the VPS, SPS, or PPS may be sent in-band with the coded bitstream. In some embodiments, the VPS, SPS, or PPS may be sent out-of-band in a transmission separate from the NAL units containing the encoded video data.

[0106] The video bitstream may also include a supplemental enhancement information (SEI) message. For example, a SEI NAL unit may be part of a video bitstream. In some cases, the SEI message may contain information that is not required for the decoding process. For example, the information in the SEI message may not be necessary for the decoder to decode the video picture of the bitstream, but the decoder may use the information to improve the display or processing of the picture (e.g., decoded output). The information in the SEI message may be embedded metadata. In an illustrative example, the decoder side entity may use the information in the SEI message to improve the visibility of the content. In some instances, certain application standards may mandate the presence of such SEI messages in the bitstream, so that quality improvements can be brought to all devices that comply with the application standard (e.g., the carrying of frame packing SEI messages for frame-compatible planar stereoscopic 3DTV video formats, the case where each frame of the video carries an SEI message, processing recovery point SEI messages, using pan scan rectangle SEI messages in DVB, and many other examples).

[0107] The output 110 of the encoding device 104 may send the NAL units constituting the encoded video bitstream data to the decoding device 112 of the receiving device via a communication link 120. The input 114 of the decoding device 112 may receive the NAL units. The communication link 120 may include a channel provided by a wireless network, a wired network, or a combination of wired and wireless networks. The wireless network may include any wireless interface or combination of wireless interfaces and may include any suitable wireless network (e.g., the Internet or other wide area network, a packet-based network, a WiFi, or a wireless network). TM , Radio Frequency (RF), UWB, WiFi-Direct, Cellular, Long Term Evolution (LTE), WiMax TM The wired network may include any wired interface (e.g., optical fiber, Ethernet, Ethernet over power line, Ethernet over coaxial cable, digital signal line (DSL), etc.). Various equipment (e.g., base stations, routers, access points, bridges, gateways, switches, etc.) may be used to implement wired networks and / or wireless networks. The encoded video bitstream data may be modulated according to a communication standard such as a wireless communication protocol and sent to a receiving device.

[0108] In some examples, the encoding device 104 may store the encoded video bitstream data in the storage 108. The output 110 may retrieve the encoded video bitstream data from the encoder engine 106 or from the storage 108. The storage 108 may include any of a variety of distributed or locally accessed data storage media. For example, the storage 108 may include a hard drive, a storage disk, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing the encoded video data. The storage 108 may also include a decoded picture buffer (DPB) for storing reference pictures for use in inter-frame prediction. In another example, the storage 108 may correspond to a file server or another intermediate storage device that can store the encoded video generated by the source device. In this case, the receiving device including the decoding device 112 can access the stored video data from the storage device via streaming or downloading. The file server can be any type of server capable of storing encoded video data and sending the encoded video data to the receiving device. Example file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The receiving device can access the encoded video data through any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both suitable for accessing the encoded video data stored on the file server. The transmission of the encoded video data from storage 108 may be a streaming transmission, a download transmission, or a combination thereof.

[0109] The input 114 of the decoding device 112 receives the encoded video bitstream data, and the video bitstream data can be provided to the decoder engine 116, or to the storage 118 for later use by the decoder engine 116. For example, the storage 118 may include a DPB for storing reference pictures for use in inter-frame prediction. A receiving device including the decoding device 112 can receive the encoded video data to be decoded via the storage 108. The encoded video data can be modulated according to a communication standard such as a wireless communication protocol and sent to the receiving device. The communication medium for sending the encoded video data may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other equipment that can be used to facilitate communication from a source device to a receiving device.

[0110] The decoder engine 116 can decode the encoded video bitstream data by entropy decoding (e.g., using an entropy decoder) and extracting elements of one or more coded video sequences that constitute the encoded video data. The decoder engine 116 can then rescale the encoded video bitstream data and perform an inverse transform. The residual data is then passed to the prediction stage of the decoder engine 116. The decoder engine 116 then predicts a block of pixels (e.g., a PU). In some examples, the prediction is added to the output of the inverse transform (residual data).

[0111] The decoding device 112 may output the decoded video to a video destination device 122, which may include a display or other output device for displaying the decoded video data to a consumer of the content. In some aspects, the video destination device 122 may be part of a receiving device that includes the decoding device 112. In some aspects, the video destination device 122 may be part of a separate device other than the receiving device.

[0112] In some embodiments, the video encoding device 104 and / or the video decoding device 112 can be integrated with the audio encoding device and the audio decoding device, respectively. The video encoding device 104 and / or the video decoding device 112 may also include other hardware or software necessary to implement the above-mentioned coding and decoding techniques, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. The video encoding device 104 and the video decoding device 112 can be integrated as part of a combined encoder / decoder (codec) in the corresponding device. Refer to the following Fig.10 An example of specific details of the encoding device 104 is described below. Fig.11 An example describing specific details of the decoding device 112.

[0113] Figure 1The example system shown is an illustrative example that can be used in this article. The technology of processing video data using the technology described in this article can be performed by any digital video encoding and / or decoding device. Although the technology of the present disclosure is usually performed by a video encoding device or a video decoding device, these technologies can also be performed by a combined video encoder-decoder (commonly referred to as a "codec"). In addition, the technology of the present disclosure can also be performed by a video preprocessor. The source device and the receiving device are merely examples of such codec devices, wherein the source device generates the encoded and decoded video data for transmission to the receiving device. In some examples, the source device and the receiving device can operate in a substantially symmetrical manner so that each device includes a video encoding and decoding component. Therefore, the example system can support one-way or two-way video transmission between video devices, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0114] Extensions to the HEVC standard include a multi-view video codec extension (referred to as MV-HEVC) and a scalable video codec extension (referred to as SHVC). The MV-HEVC and SHVC extensions share the concept of layered coding, including different layers in the coded video bitstream. Each layer in the coded video sequence is addressed by a unique layer identifier (ID). The layer ID may be present in the header of the NAL unit to identify the layer associated with the NAL unit. In MV-HEVC, different layers may represent different views of the same scene in the video bitstream. In SHVC, different scalable layers are provided that represent the video bitstream with different spatial resolutions (or picture resolutions) or different reconstruction fidelity. The scalable layer may include a base layer (layer ID = 0) and one or more enhancement layers (layer ID = 1, 2, ... n). The base layer may conform to the profile of the first version of HEVC and represent the lowest available layer in the bitstream. Compared to the base layer, the enhancement layer has increased spatial resolution, temporal resolution or frame rate and / or reconstruction fidelity (or quality). Enhancement layers are organized hierarchically and may (or may not) depend on lower layers. In some examples, different layers may be encoded using a single standard codec (e.g., all layers are encoded using HEVC, SHVC, or other codec standards). In some examples, different layers may be encoded using multiple standard codecs. For example, a base layer may be encoded using AVC, while one or more enhancement layers may be encoded using SHVC and / or MV-HEVC extensions of the HEVC standard.

[0115] Typically, a layer consists of a set of VCL NAL units and a corresponding set of non-VCL NAL units. NAL units are assigned specific layer ID values. Layers can be hierarchical in the sense that layers can depend on lower layers. A layer set refers to a set of self-contained layers represented in a bitstream, meaning that layers in a layer set can depend on other layers in the layer set during decoding, but do not depend on any other layers for decoding. Therefore, the layers in a layer set can form an independent bitstream that can represent video content. The set of layers in a layer set can be obtained from another bitstream through the operation of a sub-bitstream extraction process. A layer set may correspond to a set of layers to be decoded when a decoder wants to operate according to certain parameters.

[0116] The picture order count (POC) can be used in video codec standards to identify the display order of pictures. Although there are cases where two pictures in a coded video sequence may have the same POC value, it does not usually occur within a coded video sequence. When multiple coded video sequences are present in the bitstream, pictures with the same POC value can be closer to each other in decoding order. The POC value of a picture can be used for reference picture list construction, derivation of reference picture sets such as in HEVC, and motion vector scaling.

[0117] As described above, various prediction modes can be used in the video encoding and decoding process, including intra-frame prediction and inter-frame prediction. As described above, the video codec can divide the codec block (e.g., CTU) according to a tree structure (e.g., a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure) to generate two or more sub-blocks, wherein the two or more sub-blocks can be independently encoded and decoded. For example, different sub-blocks generated by the codec block divided using the tree structure can be encoded and decoded using a plane mode, a DC mode, etc., without inheriting the common prediction mode requirements for different sub-blocks.

[0118] In some examples, intra-subpartitioning or intra-subpartitioning (ISP) mode may be used as a codec tool to partition a codec block into two or more partitions. In some cases, the ISP mode may be used in addition to a regular or tree partitioning structure. For example, a picture may be partitioned into codec units or blocks using a QTBT structure or an MTT structure, and the codec units or codec blocks may be further partitioned using ISP. In ISP mode, two or more partitions resulting from partitioning a codec block share the same intra prediction mode. For example, two or more partitions of a codec block may be separately encoded or decoded (e.g., sequentially), but the same intra prediction mode is used for each of the two or more partitions. For example, in some implementations, if the ISP mode is enabled or selected to be applied to a codec block, the codec block may be partitioned and the same intra prediction mode may be applied to each partition.

[0119] In various examples, the ISP mode may divide a codec block, such as a luma intra prediction block, into two or four ISP partitions (or ISP subblocks or subblocks), depending on the size or dimension of the codec block. In some examples, the division may be vertical or horizontal based on the dimension of the codec block. The number of ISP subblocks into which the codec block is partitioned may be based on the number of samples that will exist in the ISP subblock. For example, in some implementations, the ISP subblock may need to have a minimum number of samples, such as 16 samples or other number of samples. Therefore, if partitioning the block into four ISP subblocks may result in each of the four subblocks having at least a minimum number of samples (e.g., 16 samples), the ISP mode may decide to partition the block into four ISP subblocks. However, if partitioning the block into four ISP subblocks may not result in each of the four subblocks having at least a minimum number of samples (e.g., 16 samples), the ISP mode may partition the block into two ISP subblocks if each of the two subblocks has at least a minimum number of samples (e.g., 16 samples). The ISP sub-block will be referred to as a sub-block hereinafter, which may be different from a sub-block that may be generated using a regular or tree partitioning structure (eg, using a QTBT structure or an MTT structure), as described above.

[0120] Figure 2 is a block diagram showing an example of partitioning a codec block into sub-blocks. The current block 202 may be partitioned into a plurality of sub-blocks in the ISP mode. The sub-blocks generated using the ISP partitioning block are also referred to as ISP sub-blocks or ISP blocks. The current block 202 is shown to have dimensions of width W and height H. In one example, the current block 202 may have a plurality of samples corresponding to H×W. For example, a current block 202 of dimension 4×8 may have 32 samples. If the current block 202 is partitioned into four sub-blocks using the ISP mode, each of the four resulting sub-blocks will have a quarter of the 32 samples, i.e., 8 samples per sub-block. As described above, in the case where the minimum number of samples is 16, such a division would violate the requirement that each sub-block has at least 16 samples. Therefore, for the above values ​​of H and W, the current block 202 is partitioned into two sub-blocks, each of which has half of the 32 samples (i.e., 16 samples), satisfying the minimum sample requirement. It should be noted that, for similar reasons, a block of dimension 8×4 may also be divided into two sub-blocks to meet the minimum sample requirement of 16 samples in each sub-block.

[0121] like Figure 2As shown, the horizontal ISP partitioning of the current block 202 may generate two ISP blocks 204a and 204b, each of which has the same width (W) as the current block 202 and a half height (H / 2) of the current block 202. Similarly, the vertical ISP partitioning of the current block 202 may generate two ISP blocks 206a and 206b, each of which has the same height (H) as the current block 202 and a half width (W / 2) of the current block 202.

[0122] Figure 3 is a block diagram showing an example of partitioning a codec block into four sub-blocks. A current block 302 may be divided into four sub-blocks or ISP blocks in ISP mode. The current block 302 is shown as having dimensions of width W and height H. The example of partitioning the current block 302 into four sub-blocks may be applicable to dimensions of the current block 302 that, when partitioned into four sub-blocks, may produce at least 16 samples in each resulting sub-block. For example, in the case where the minimum number of samples per sub-block is 16, the current block 302 may have dimensions other than those described above with respect to Figure 2 Dimensions other than 4×8 or 8×4 (e.g., dimensions greater than 4×8 or 8×4). The 4×4 dimension of the current block 302 (which in some embodiments will be the minimum allowed size of a codec block) may also be excluded from being partitioned into sub-blocks using the ISP mode.

[0123] In one example, the horizontal ISP partitioning of the current block 302 may produce four ISP blocks 304a, 304b, 304c, and 304d, each ISP block having the same width (W) as the current block 202 and a height (H / 4) that is one-quarter of the current block 202. Similarly, the vertical ISP partitioning of the current block 302 may produce four ISP blocks 306a, 306b, 306c, and 306d, each ISP block having the same height (H) as the current block 202 and a width (W / 4) that is one-quarter of the current block 302.

[0124] In an example implementation of the ISP mode, once the number of partitions (e.g., two or four) and the ISP partition type indicating the partition type (e.g., horizontal or vertical partition) have been determined for a block, the current block or codec block can be partitioned accordingly, and the resulting ISP sub-blocks can be processed sequentially. For example, two or more sub-blocks resulting from applying the ISP mode to a codec block can be processed sequentially. As previously discussed, processing the sub-blocks can include predicting each sub-block using intra-frame prediction, applying a transform codec, and then reconstructing the sub-block.

[0125] In some implementations, various syntax elements may be used to encode and signal the ISP mode. The syntax table for encoding and decoding a unit using the ISP mode is shown in Table 2 and is further discussed below. According to an example implementation, one bit of the codec block (in some cases referred to as intra_subpartitions_mode_flag[x0][y0]) may be used to signal whether the codec block is split into ISP blocks. A second bit (in some cases referred to as intra_subpartitions_split_flag[x0][y0]) may be used to indicate the type of ISP splitting to be performed on the codec block, for example, whether the splitting type is horizontal or vertical.

[0126] In some examples, different categories of processing order can be used to process ISP blocks obtained by segmenting a codec block using an ISP mode. The processing order can be based on the intra prediction mode and the segmentation type used for the codec block. For example, when processing two or more ISP blocks obtained from a segmented codec block, two different categories of processing order (referred to as normal order and reverse order) can be used.

[0127] In normal order, the first subpartition to be processed is the one containing the top left sample of the codec block, and then it continues downwards (in case of horizontal partitioning) or to the right (in case of vertical partitioning). Figure 2 In , the normal processing order for horizontal splitting may include first processing ISP block 204a and then proceeding downward to process ISP block 204b. Figure 2 In , the normal processing order for vertical segmentation may include processing ISP block 206a first, and then continuing to the right to process ISP block 206b. Figure 3 In, the normal processing order for horizontal splitting may include first processing ISP block 304a, and then continuing downward to process ISP blocks 304b, 304c, and 304d in sequence. Figure 3 , the normal processing order for vertical segmentation may include first processing ISP block 306a and then continuing to the right to sequentially process ISP blocks 306b, 306c, and 306d.

[0128] The reverse processing order may include starting from the sub-partition containing the lower left samples of the codec block and continuing upward (in the case of horizontal partitioning) or starting from the sub-partition containing the upper right samples of the codec block and continuing to the left (in the case of vertical partitioning). Figure 2 In , the reverse processing order for horizontal splitting may include first processing ISP block 204b and then proceeding upward to process ISP block 204a. Figure 2In , the reverse processing order for vertical segmentation may include first processing ISP block 206b and then continuing to the left to process ISP block 206a. Figure 3 , the reverse processing order for horizontal splitting may include first processing ISP block 304d and then continuing upward in sequence to process ISP blocks 304c, 304b, and 304a. Figure 3 , the reverse processing order for vertical segmentation may include first processing ISP block 306d and then continuing to the left to sequentially process ISP blocks 306c, 306b, and 306a.

[0129] A variant of ISP using only normal processing order is used in JVET Working Draft 4 (WD4, available from http: / / phenix.it-sudparis.eu / jvet / doc_end_user / documents / 13_Marrakech / wg11 / JVET-M1001-v1.zip), the entire contents of which are incorporated herein by reference and for all purposes. It should be noted that the terms sub-block (as used with respect to ISP), sub-partition (as used with respect to ISP), and ISP block are used interchangeably herein, and all of these terms refer to blocks obtained by partitioning a codec block using the ISP mode.

[0130] An example of syntax and semantics associated with ISP mode in JVET WD4 is shown in Table 2 below. The syntax emphasized using italic text corresponds to the ISP mode for the codec unit located at the (x,y) coordinates [x0][y0] of the picture. The reference line index selected by the encoder for the reference line candidate to predict the codec unit is indicated by the syntax element intra_luma_ref_idx. The syntax elements cbWidth and cbHeight correspond to the width and height dimensions of the codec unit (e.g., as Figure 2 and Figure 3 The width W and height H for the corresponding current blocks 202 and 203 are shown). The variable MaxTbSizeY refers to the maximum transform block size and is used as a size threshold for the cbWidth and cbHeight dimensions. As previously described, intra_subpartitions_mode_flag[x0][y0] is a syntax element used to signal whether the ISP mode is used to split the codec unit into sub-blocks (or ISP blocks). The syntax element intra_subpartitions_split_flag[x0][y0] is used to indicate the ISP split type (the type of split to be performed), for example, whether horizontal splitting or vertical splitting is used to split the codec block.

[0131]

[0132]

[0133] Table 2 - Codec unit syntax table

[0134] As can be seen from Table 2, when at least the condition (cbWidth<=MaxTbSizeY||cbHeight<=MaxTbSizeY) is satisfied, that is, when either cbWidth or cbHeight (or both) is less than MaxTbSizeY, intra_subpartitions_mode_flag[x0][y0] is signaled. On the other hand, when at least the conditions cbWidth<=MaxTbSizeY and cbHeight<=MaxTbSizeY are satisfied, that is, when both cbWidth and cbHeight are less than MaxTbSizeY, intra_subpartitions_split_flag[x0][y0] is signaled. Thus, in an implementation using the semantics specified in Table 2, when one (but not both) of cbWidth or cbHeight is greater than MaxTbSizeY, only intra_subpartitions_mode_flag[x0][y0] is signaled, and intra_subpartitions_split_flag[x0][y0] is not signaled. As will be explained further below, when intra_subpartitions_mode_flag[x0][y0] is not signaled, the value of intra_subpartitions_mode_flag[x0][y0] is inferred to be equal to 0; and similarly, when intra_subpartitions_split_flag[x0][y0] is not signaled, the value of intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 0.

[0135] In addition to intra_subpartitions_mode_flag[x0][y0] and intra_subpartitions_split_flag[x0][y0], the variable IntraSubPartitionsSplitType is also defined, which specifies the split type used for the current luma codec block. The values ​​of IntraSubPartitionsSplitType and the corresponding names are associated as shown in Table 3 below.

[0136] IntraSubPartitionsSplitType Name of IntraSubPartitionsSplitType 0 ISP_NO_SPLIT 1 ISP_HOR_SPLIT 2 ISP_VER_SPLIT

[0137] Table 3 – Name associations for IntraSubPartitionsSplitType

[0138] As can be seen from Table 3, IntraSubPartitionsSplitType is a variable used to indicate whether there is no ISP split (IntraSubPartitionsSplitType = 0), whether horizontal ISP split (IntraSubPartitionsSplitType = 1) is applied, or whether vertical ISP split (IntraSubPartitionsSplitType = 2) is applied. When there is no ISP split (IntraSubPartitionsSplitType = 0), for example, according to QTBT, MTT, etc., the rule transformation tree parsing structure is used to divide the block. For the case of horizontal or vertical ISP split, the variable NumIntraSubPartitions is used to indicate the number of sub-blocks or ISP blocks (for example, Figure 2 2 ISP blocks in or Figure 3 For the transform unit corresponding to the coding unit, the corresponding number of sub-blocks are parsed in sequence. Table 4 below provides the syntax of the transform tree using the ISP mode.

[0139]

[0140] Table 4 - Syntax table of transformation tree

[0141]

[0142] Based on the above description of syntax elements and variables associated with ISP mode, the following semantics can be used to apply ISP to a codec unit:

[0143] intra_subpartitions_mode_flag[x0][y0] equal to 1 specifies that the current intra codec unit is divided into NumIntraSubPartitions[x0][y0] rectangular transform block subpartitions. intra_subpartitions_mode_flag[x0][y0] equal to 0 specifies that the current intra codec unit is not divided into rectangular transform block subpartitions.

[0144] When intra_subpartitions_mode_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0145] intra_subpartitions_split_flag[x0][y0] specifies whether the intra subpartition split type is horizontal or vertical. When intra_subpartitions_split_flag[x0][y0] is not present, it is inferred to be equal to 0.

[0146] The variable IntraSubPartitionsSplitType specifies the split type used for the current luma codec block, as shown in Table 3. IntraSubPartitionsSplitType is derived as follows:

[0147] – If intra_subpartitions_mode_flag[x0][y0] is equal to 0, IntraSubPartitionsSplitType is set equal to 0.

[0148] – Otherwise, IntraSubPartitionsSplitType is set equal to 1 + intra_subpartitions_split_flag[x0][y0].

[0149] As can be seen from the above semantics, a value of intra_subpartitions_mode_flag[x0][y0] equal to 1 can be used to specify that the codec unit is divided into a number of sub-blocks indicated by the variable NumIntraSubPartitions[x0][y0], where the sub-blocks are, for example Figure 2 and Figure 3 ISP block shown in FIG. 10 . A value of intra_subpartitions_mode_flag[x0][y0] equal to 0 may be used to specify that the codec unit is not divided into rectangular transform block subpartitions using ISP mode. In some implementations, when intra_subpartitions_mode_flag[x0][y0] is not present for a codec unit, the value of intra_subpartitions_mode_flag[x0][y0] for the codec unit is inferred to be equal to 0.

[0150] As previously described, intra_subpartitions_split_flag[x0][y0] may be used to specify whether the ISP split type is horizontal or vertical. In some implementations, when intra_subpartitions_split_flag[x0][y0] is not present for a codec unit, the value of intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 0.

[0151] As described above with reference to Table 3, the variable IntraSubPartitionsSplitType specifies the type of partitioning used for the codec block. The value of this variable IntraSubPartitionsSplitType can be derived as follows: if the value of intra_subpartitions_mode_flag[x0][y0] is equal to 0, then IntraSubPartitionsSplitType is set equal to 0; otherwise, the value of IntraSubPartitionsSplitType is set equal to 1+intra_subpartitions_split_flag[x0][y0].

[0152] In some implementations, the variable NumIntraSubPartitions used to specify the number of sub-blocks into which the codec unit is divided can be derived as follows: If the variable IntraSubPartitionsSplitType has a value equal to 0, indicating ISP_NO_SPLIT, then the value of the variable NumIntraSubPartitions is set to be equal to 1 (1 is the default value indicating that no ISP is applied to the codec unit). Otherwise, if the variable IntraSubPartitionsSplitType has a value equal to 1 (horizontal split ISP_HOR_SPLIT) or 2 (vertical split ISP_VER_SPLIT), as described in reference Figure 2 and Figure 3As explained, the value of the variable NumIntraSubPartitions is set equal to 2 or 4 based on the dimension of the codec unit or the current block. For example, if the current block 202 is a 4×8 block (cbWidth equal to 4, cbHeight equal to 8) or an 8×4 block (cbWidth equal to 8, cbHeight equal to 4), NumIntraSubPartitions is set to 2 to implement the splitting of the current block 202 into two ISP blocks, each with a minimum number of 16 samples, based on the horizontal or vertical splitting indicated by the variable IntraSubPartitionsSplitType. On the other hand, if the dimension of the codec unit is not (greater than) 8×4, 4×8, and 4×4 block sizes, NumIntraSubPartitions is set equal to 4 (for example, as referred to in reference to the splitting of the current block 202 into two ISP blocks, each with a minimum number of 16 samples, based on the horizontal or vertical splitting indicated by the variable IntraSubPartitionsSplitType). Figure 3 The current block 302 is split into four ISP blocks as discussed).

[0153] As described in reference to Tables 2-4, there are several issues in the implementation of the ISP mode based on the above semantics and syntax. In one example (e.g., based on VVC), as discussed above with reference to the transform tree syntax structure of Table 4, when the width (cbWidth) or height (cbHeight) of the luma codec block (CB) is greater than the maximum transform block size (expressed as MaxTbSizeY, where Y indicates luma samples), the codec block is split into two or more transform units. Note that some implementations may include a fixed value for MaxTbSizeY (e.g., JVET WD4 specifies a fixed value of 64 for MaxTbSizeY). For some implementations or future standards, it is possible to allow the value of MaxTbSizeY to be another fixed value (e.g., a value of 32 or other value) or to be a variable. In some examples, the value of MaxTbSizeY may be signaled in the bitstream, or may be derived from other syntax elements in the bitstream. Therefore, when determining the signaling of the above syntax elements, the comparison of cbWidth and cbHeight with MaxTbSizeY can be based on a fixed value or a variable value of MaxTbSizeY.

[0154] As previously described, when the codec block width (cbWidth) or the codec block height (cbHeight) is less than or equal to the maximum transform block size MaxTbSizeY, or if both cbWidth and cbHeight are less than or equal to MaxTbSizeY, the mode flag intra_subpartitions_mode_flag of the ISP mode is signaled for the codec block. Therefore, in the example where one of cbWidth or cbHeight is greater than MaxTbSizeY and the other is less than MaxTbSizeY, intra_subpartitions_mode_flag is signaled (e.g., with a value indicating that ISP mode is enabled for the codec block, such as with a value of 1). In this implementation (referring back to the syntax of the codec unit shown in Table 2), since one of cbWidth or cbHeight is greater than MaxTbSizeY, the syntax element intra_subpartitions_split_flag of the split flag is not signaled.

[0155] When intra_subpartitions_split_flag is not signaled for a codec block, according to existing implementations, intra_subpartitions_split_flag is inferred to be equal to 0. According to the semantics provided above, the split type of the codec block is provided by the value of the variable IntraSubPartitionsSplitType, which is set equal to 1+intra_subpartitions_split_flag. Therefore, when intra_subpartitions_split_flag is inferred to be 0, the variable IntraSubPartitionsSplitType is inferred to be equal to 1. From Table 3 showing the name association of IntraSubPartitionsSplitType, IntraSubPartitionsSplitType has a value of 1 corresponding to horizontal split (ISP_HOR_SPLIT). As previously described, the derivation of IntraSubPartitionsSplitType equal to 1 (corresponding to horizontal split) is for the case where one of cbWidth or cbHeight is greater than MaxTbSizeY and the other is less than MaxTbSizeY. It is worth noting that this deduction that the split type is a horizontal split is the same whether cbWidth is greater than MaxTbSizeY (and cbHeight is less than MaxTbSizeY) or cbHeight is greater than MaxTbSizeY (and cbWidth is less than MaxTbSizeY). The value of IntraSubPartitionsSplitType being deduced to be equal to 1 regardless of which of the two dimensions of the codec block (cbWidth or cbHeight) is greater than MaxTbSizeY may lead to undesirable effects.

[0156] For example, when intra_subpartitions_mode_flag is equal to 1 and the cbWidth or cbHeight of the codec block is greater than MaxTbSizeY, for some values ​​of cbWidth or cbHeight, the partition type is always derived as horizontal which may not be the best choice of partition type. To illustrate, consider an example where MaxTbSizeY is equal to 32, cbWidth is equal to 64, and cbHeight is equal to 32. In this example, the shape of the codec block will be similar to Figure 2The shape of the current block 202 shown, where the width W is greater than the height H. In this case, ISP vertical splitting would be the best choice, which can produce sub-blocks with each width and height less than or equal to MaxTbSizeY (similar to the shape of ISP blocks 206a and 206b). For example, ISP vertical splitting of a codec block with cbWidth equal to 64 and cbHeight equal to 32 into two sub-partitions will produce two sub-blocks, each with a width equal to 32 and a height equal to 32, where the width and height are equal to MaxTbSizeY, which is 32. However, in the current implementation as explained above, based on the intra_subpartitions_split_flag being inferred to have a value of 0 and correspondingly IntraSubPartitionsSplitType being inferred to be equal to 1, ISP horizontal splitting will be inferred. The ISP horizontal split in the example of a codec block with cbWidth equal to 64 and cbHeight equal to 32 into two sub-partitions would result in two sub-blocks, each with width equal to 64 and height equal to 16 (similar to the shape of ISP blocks 204a and 204b), where the height of both sub-blocks would violate the requirement of MaxTbSizeY being greater than 32. Therefore, a solution is needed that can overcome the above-mentioned undesirable consequences associated with inferring a horizontal split decision regardless of the specific values ​​of cbWidth and cbHeight of the codec block.

[0157] Another problem in existing implementations of the ISP mode may correspond to one or more dimensions of the sub-blocks obtained by splitting the codec block. Consider another illustrative example to explain the problem associated with the width of the sub-block being greater than the maximum size threshold. In such an example, the following is assumed for the codec block: MaxTbSizeY is equal to 16, cbWidth is equal to 32, and cbHeight is equal to 64. As previously described, this combination of values ​​will result in both cbWidth and cbHeight being greater than MaxTbSizeY, which will result in a decision to enable ISP mode to split the codec block. Since at least one of cbWidth and cbHeight is greater than MaxTbSizeY (in this case, in fact both cbWidth and cbHeight are greater than MaxTbSizeY), in the current implementation, intra_subpartitions_split_flag is not signaled and is therefore inferred to be equal to 0, resulting in IntraSubPartitionsSplitType being derived as 1, corresponding to the ISP horizontal split of the codec block. In this case, considering the block dimension as explained earlier, the number of subpartitions can be 4, so NumIntraSubPartitions is derived as 4. The four resulting sub-blocks obtained from the horizontal partition (similar to Figure 3 In the example of , four ISP blocks 304a-304d) obtained by splitting the current block 302 using horizontal partitioning will each have a width of 32 and a height of 16. Therefore, a width of 32 for each sub-block will mean that each sub-block has a width greater than MaxTbSizeY which is 16. As mentioned above, it is undesirable to have sub-blocks with a width greater than the maximum transform block size MaxTbSizeY. Accordingly, an ISP solution is also needed in which the width and height dimensions of the sub-blocks obtained by applying ISP partitioning to the codec block do not exceed a size threshold of the maximum transform block size MaxTbSizeY.

[0158] Another problem associated with existing implementations relates to the use of ISP mode in encoder or decoder designs that utilize virtual pipeline data units (VPDUs). A VPDU is a non-overlapping cell in a picture or video frame. For example, a VPDU may be a non-overlapping M×M-luminance (L) / N×N-chrominance (C) unit in a picture. In some examples of a hardware decoding process, consecutive VPDUs may be processed in parallel by multiple processing / decoding pipeline stages (e.g., different decoding pipeline stages process different VPDUs simultaneously). To support parallel processing, each VPDU may include a codec unit that can be processed independently. For example, processing stages such as reconstruction, transform, and prediction may be performed in different pipeline stages that process codec units using VPDUs. In some cases, in some pipelines, the VPDU size may be roughly proportional to the buffer size. For example, the VPDU size may be set to the size of a transform block (TB) size. In some examples, only a subset of the processing stages (e.g., transform or inverse transform) of the VPDU may be processed independently.

[0159] In VVC, design choices for the VPDU include a size of 64×64 luma samples, where the maximum transform block size MaxTbSizeY can be 64 samples. When a partitioning scheme is used, some implementations may include one or more constraints that prohibit partitioning boundaries from crossing VPDU boundaries in order to support the above-mentioned independent processing capabilities. In an illustrative example in which the size of the VPDU is 64×64 samples each, the VPDU can be implemented in hardware using a pipeline capable of processing 64×64 sample units. In such an example, one or more constraints may prevent the partitioning of codec units larger than 64×64 samples. Such constraints can ensure that only codec blocks that can be accommodated in the VPDU without being partitioned are allowed to be processed using the VPDU. JVET-L1002 includes examples of these and other constraints in which VPDUs are used to partition codec units.

[0160] In current implementations of ISP mode, it is possible to have codec units whose size exceeds the 64×64 boundary. For example, some codec units of size 64×128 and 128×64 are supported in VVC. In some cases, ISP mode can be enabled to indicate ISP segmentation for such codec units that are larger than the 64×64 boundary. For example, if ISP segmentation is applied to such codec units of 64×128 or 128×64 to produce sub-blocks smaller than the 64×64 boundary, such sub-blocks can be accommodated within the VPDU for processing. As described above, different sub-blocks of the same codec unit may not support independent processing in different VPDUs. Therefore, processing different sub-blocks (obtained by dividing the same codec unit) in separate VPDUs is undesirable and may violate restrictions on codec units that cross VPDU boundaries. Since current implementations can allow codec units of size larger than 64×64 to be encoded and decoded using ISP mode, a solution that does not violate the VPDU size constraints when ISP mode is enabled is needed.

[0161] Systems, methods, and computer-readable media for improving video coding and decoding are described herein. For example, the systems, methods, and computer-readable media described herein improve the design and use of intra-frame sub-block partitioning (ISP) mode and address the above-mentioned needs in existing implementations. The various features described herein may be used alone or in any suitable combination. As previously mentioned, the terms intra-frame sub-block partitioning (ISP), sub-partitioning (with respect to ISP mode use), sub-block intra-frame prediction (with respect to ISP mode use), and other similar terms may be used interchangeably herein.

[0162] In some examples of implementing the ISP mode, the semantics and / or syntax of the intra-subpartition split flag for the codec block can be provided in a manner to ensure that the optimal value of the intra-subpartition split flag (also referred to herein as the subpartition split flag) is determined for the codec block. An illustrative example of an intra-subpartition split flag is the intra_subpartitions_split_flag mentioned above. For example, in the case where one of the above-mentioned cbWidth or cbHeight is greater than MaxTbSizeY (also referred to herein as the "size threshold"), even though the intra_subpartitions_split_flag may not exist, the value of the intra_subpartitions_split_flag is inferred to be a value that can result in the optimal derivation of the split type variable IntraSubPartitionsSplitType. In some examples, depending on whether cbWidth is greater than MaxTbSizeY or cbHeight is greater than MaxTbSizeY, the value of intra_subpartitions_split_flag is inferred so that the IntraSubPartitionsSplitType derived using the inferred value of intra_subpartitions_split_flag indicates ISP vertical split or ISP horizontal split, respectively. An illustrative example showing changes to the existing syntax and / or semantics of JVET WD4 of VVC to obtain the inference of the above-mentioned intra_subpartitions_split_flag is provided below.

[0163] In some examples, a partition structure for ISP mode may be provided to ensure that the dimensions of the sub-blocks obtained by dividing the codec block do not violate a size threshold such as MaxTbSizeY. For example, an ISP mode may be provided in which any dimension (width or height) of any sub-block does not exceed the value of MaxTbSizeY. In one example, when the sub-partition split type is horizontal or vertical, a recursive structure similar to the transform_tree() syntax structure of Table 4 (also referred to herein as the "transform tree syntax structure") may be implemented. In another example, when the cbWidth or cbHeight (or both cbWidth and cbHeight) of the codec block decoded with ISP is greater than MaxTbSizeY, the value of the variable NumIntraSubPartitions (also referred to herein as the "number of sub-partitions") is modified so that the width or height of the sub-block does not exceed MaxTbSizeY. In another example, the semantics or syntax of intra_subpartitions_mode_flag (also referred to herein as the "sub-partition mode flag") is provided to ensure that the dimensions (width or height or both) of the sub-block do not exceed MaxTbSize. In another example, when the cbWidth or chHeight of the codec block is greater than MaxTbSizeY, the codec block is restricted to be encoded using the ISP codec. An example showing how to change the existing syntax and / or semantics of JVET WD4 of VVC in the above manner to restrict the sub-block size is further described below.

[0164] In some examples of enabling ISP mode, one or more size constraints may be implemented for codecs that may be processed using VPDUs. In some examples, one or more size constraints prevent ISP mode from being enabled for codecs that cross VPDU boundaries. An example of additional existing syntax and / or semantics for JVET WD4 of VVC is provided below.

[0165] Figure 4is a block diagram illustrating an example of a codec block that can be encoded and decoded with an ISP using the example syntax and / or semantics described herein. In some examples, various decisions regarding enabling the ISP mode, the type of segmentation (horizontal or vertical) to be applied if the ISP mode is enabled, the number of ISP sub-partitions, etc. can be based at least in part on one or more dimensions of the codec block. The codec block 402 is shown with dimensions cbWidth and cbHeight corresponding to the width and height of the codec block 402, respectively. The codec block 402 may include a codec block such as a CU or PU (or CB or PB) of a picture that can be processed using the ISP mode. In some examples, the number of samples (e.g., luma samples) in the codec block 402 can be proportional to the size of the codec block 402 obtained by cbWidth×cbHeight. In some examples, the ISP mode can be used to segment the codec block 402 based on the relationship between the dimensions cbWidth and cbHeight and the corresponding size thresholds.

[0166] In some examples, the transform blocks used to partition the codec block according to the transform tree structure (or other partitioning structure) can be square-shaped, where the maximum height and width of the transform block are the same, referred to as the maximum transform block size MaxTbSizeY. In some examples, applying the ISP mode to partition the codec block 402 can be based on the cbWidth and cbHeight dimensions relative to MaxTbSizeY. In some examples, the size of the codec block 402 that can be partitioned by partitioning the codec block 402 (e.g., similar to MaxTbSizeY) can also be limited based on MaxTbSizeY. Figure 2 and Figure 3 ISP segmentation shown in , wherein the techniques described herein are used to ensure that the sub-blocks obtained from the segmented codec block 402 also do not violate the corresponding size threshold. In some examples where the codec block 402 can be processed using a VPDU, one or more constraints on the cbWidth and cbHeight dimensions of the codec block 402 can be implemented relative to the VPDU boundaries (e.g., based on the maximum width and / or height of the VPDU). These and other examples are discussed in further detail below.

[0167] As previously described, there may be situations where intra_subpartitions_split_flag may not be signaled for a codec block that is specified to be coded using ISP mode. In some examples, if an intra-subpartition split flag (e.g., intra_subpartitions_split_flag) of the codec block 402 is not present, then based on cbWidth and / or cbHeight relative to a size threshold such as a maximum transform block size MaxTbSizeY, the value of the intra-subpartition split flag (e.g., intra_subpartitions_split_flag) may be inferred to be a first value or a second value. In some examples, the type of split (vertical or horizontal) to be applied to the codec block 402 may be based on the first value or the second value.

[0168] In some examples, when cbWidth or cbHeight is less than or equal to MaxTbSizeY or if both cbWidth and cbHeight are less than or equal to MaxTbSizeY, an ISP mode flag (e.g., intra_subpartitions_mode_flag) of the codec block 402 may be signaled. Thus, in examples where one of cbWidth or cbHeight is greater than MaxTbSizeY and the other is less than MaxTbSizeY, intra_subpartitions_mode_flag is signaled with a value (e.g., a value of 1 indicating that ISP mode is enabled for the codec block, or a value of 0 indicating that ISP mode is not performed for the codec block). In this implementation, since one of cbWidth or cbHeight is greater than MaxTbSizeY, the intra-subpartition split flag intra_subpartitions_split_flag may not be present or may not be signaled.

[0169] In some examples, when the ISP mode for the codec block 402 is enabled (e.g., when intra_subpartitions_mode_flag is signaled), and the intra-subpartition split flag (e.g., intra_subpartitions_split_flag) is not present, then if cbWidth is greater than MaxTbSizeY, the codec block 402 may infer that the intra-subpartition split flag (e.g., intra_subpartitions_split_flag) is equal to a first value (e.g., a value of 1). In some examples, applying a formula for the split type obtained by IntraSubPartitionsSplitType being equal to 1+intra_subpartitions_split_flag (i.e., IntraSubPartitionsSplitType being equal to 2) may indicate that the value of intra_subpartitions_split_flag being inferred to be equal to 1 may result in a vertical split (e.g., as indicated by the name association in Table 3, an IntraSubPartitionsSplitType value of 2 corresponds to an ISP vertical split ISP_VER_SPLIT).

[0170] On the other hand, when the ISP mode for the codec block 402 is enabled (e.g., when intra_subpartitions_mode_flag is signaled), and the intra-subpartition split flag (e.g., intra_subpartitions_split_flag) is not present, then if cbHeight is greater than MaxTbSizeY, the codec block 402 may infer that the intra-subpartition split flag (e.g., intra_subpartitions_split_flag) is equal to a second value (e.g., a value of 0). In some examples, applying a formula for the split type obtained by IntraSubPartitionsSplitType equal to 1+intra_subpartitions_split_flag (i.e., IntraSubPartitionsSplitType equal to 1) may indicate that in this case the value of intra_subpartitions_split_flag is inferred to be equal to 0 which may result in a horizontal split (e.g., as shown in the name association in Table 3, an IntraSubPartitionsSplitType value of 1 corresponds to an ISP horizontal split ISP_HOR_SPLIT).

[0171] In some cases, when the ISP mode for the codec block 402 is enabled (e.g., when intra_subpartitions_mode_flag is signaled with a true value (e.g., 1)), and if neither cbHeight nor cbWidth is greater than MaxTbSizeY, the intra-subpartition split flag is signaled, and as described above, the value of intra_subpartitions_split_flag indicates whether the partition type is vertical or horizontal.

[0172] An example showing changes to the existing syntax and / or semantics of JVET WD4 for VVC with respect to intra_subpartitions_split_flag is now described. New additions to the specification are shown in italic text, and strikethrough text shows text removed from the specification. The following syntax is used to describe the above implementation for the codec block 402 present at coordinates [x0][y0], with changes to the existing implementation highlighted in italic text and strikethrough text highlighting removals from the specification.

[0173] intra_subpartitions_split_flag[x0][y0] specifies whether the intra subpartition split type is horizontal or vertical. When _flag[x0][y0] is not present, it is inferred as follows:

[0174] – If cbWidth > MaxTbSizeY, infer intra_subpartitions_split_flag[x0][y0]

[0175] is equal to 1.

[0176] – Otherwise, if cbHeight > MaxTbSizeY, intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 0.

[0177] Derive IntraSubPartitionsSplitType as follows:

[0178] – If intra_subpartitions_mode_flag[x0][y0] is equal to 0, IntraSubPartitionsSplitType is set equal to 0.

[0179] – Otherwise, IntraSubPartitionsSplitType is set equal to 1 + intra_subpartitions_split_flag[x0][y0].

[0180] As previously described, the variable NumIntraSubPartitions can be used to specify the number of sub-blocks into which the codec block 402 is divided based on the minimum sample requirement of the sub-block, intra_subpartitions_split_flag, and intra_subpartitions_mode_flag. For example, referring to Table 3, if the variable IntraSubPartitionsSplitType has a value equal to 0 (indicating ISP_NO_SPLIT), the value of the variable NumIntraSubPartitions is set to be equal to 1 (1 is the default value indicating that no ISP is applied to the codec block 402). Otherwise, if the variable IntraSubPartitionsSplitType has a value equal to 1 (horizontal split ISP_HOR_SPLIT) or a value equal to 2 (vertical split ISP_VER_SPLIT), as referred to Figure 2 and Figure 3 As explained, the value of the variable NumIntraSubPartitions is set equal to 2 or 4 based on the dimensions of the codec unit or the current block. For example, if the codec block 402 is a 4×8 block (cbWidth equals 4 and cbHeight equals 8) or an 8×4 block (cbWidth equals 8 and cbHeight equals 4), NumIntraSubPartitions is set to 2 to implement the codec block 402 being split into two ISP blocks, each having a minimum number of 16 samples, based on a horizontal or vertical split that can be indicated by the variable IntraSubPartitionsSplitType. On the other hand, if the dimensions cbWidth and cbHeight of the codec block 402 correspond to a size other than (greater than) 8×4, 4×8, and 4×4 block sizes, NumIntraSubPartitions is set equal to 4.

[0181] In another example, when the ISP mode for the codec block 402 is enabled (e.g., when intra_subpartitions_mode_flag is signaled), and the intra-subpartition split flag intra_subpartitions_split_flag is not present, then if cbWidth is greater than MaxTbSizeY and cbHeight is less than or equal to MaxTbSizeY, then intra_subpartitions_split_flag may be inferred to be equal to a first value (e.g., a value of 1) for the coding block 402. In some examples, applying a formula for the split type obtained by IntraSubPartitionsSplitType being equal to 1+intra_subpartitions_split_flag (i.e., IntraSubPartitionsSplitType being equal to 2) may indicate that in this case the value of intra_subpartitions_split_flag is inferred to be equal to 1 which may result in a vertical split (e.g., as indicated by the name association in Table 3, an IntraSubPartitionsSplitType value of 2 corresponds to an ISP vertical split ISP_VER_SPLIT).

[0182] On the other hand, when the ISP mode for the codec block 402 is enabled (e.g., when intra_subpartitions_mode_flag is signaled), and the intra-subpartition split flag intra_subpartitions_split_flag is not present, then if cbHeight is greater than MaxTbSizeY and cbWidth is less than or equal to MaxTbSizeY, then intra_subpartitions_split_flag may be inferred to be equal to a second value (e.g., a value of 0) for the codec block 402. In some examples, applying a formula for the split type obtained by IntraSubPartitionsSplitType being equal to 1+intra_subpartitions_split_flag (i.e., IntraSubPartitionsSplitType being equal to 1) may indicate that in this case the value of intra_subpartitions_split_flag is inferred to be equal to 1 which may result in a horizontal split (e.g., as shown in the name association in Table 3, an IntraSubPartitionsSplitType value of 1 corresponds to an ISP horizontal split ISP_HOR_SPLIT).

[0183] The following syntax is used to describe the above implementation for the codec block 402 present at coordinates [x0][y0], where changes to the existing implementation are highlighted in italic text and strikethrough highlights are removed from the specification.

[0184] intra_subpartitions_split_flag[x0][y0] specifies whether the intra subpartition split type is horizontal or vertical. When _flag[x0][y0] is not present, it is inferred as follows:

[0185] – If cbWidth>MaxTbSizeY and cbHeight<=MaxTbSizeY, intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 1.

[0186] – Otherwise, if cbHeight>MaxTbSizeY and cbWidth<=MaxTbSizeY, intra_subpartitions_split_flag[x0][y0] is inferred to be equal to 0.

[0187] – Otherwise, intra_subpartitions_split_flag[x0][y0] is equal to 0.

[0188] An example showing changes to the existing syntax and / or semantics of JVET WD4 for VVC defining a partition structure for ISP is now described, where the dimensions of the sub-blocks obtained from the partitioned codec block do not exceed the size threshold of MaxTbSizeY. New additions to Tables 2 and 4 are shown in italic text, and strikethrough text shows text deleted from the corresponding specification.

[0189] In a first example of defining a partition structure for the ISP mode enabled codec block 402, two or more sub-blocks may be obtained based on appropriately applying horizontal or vertical partitioning based on the above discussion of intra_subpartitions_mode_flag and intra_subpartitions_split_flag. Figure 2 and Figure 3As explained, sub-blocks or ISP blocks can be obtained. It can be tested whether the dimensions of these sub-blocks meet the requirement of being less than MaxTbSizeY. If any dimension (width or height) of the respective sub-blocks obtained from the segmentation codec block 402 (for which ISP mode is enabled) exceeds MaxTbSizeY, they are recursively divided until a sub-block whose width and height do not exceed the value of MaxTbSizeY is obtained.

[0190] In one example, when the width or height of a sub-block exceeds MaxTbSizeY, when the split type is NO_ISP_SPLIT, the sub-block may be further split using the transform_tree() structure of Table 4. On the other hand, for example, when the split type is horizontal split for a codec block whose cbHeight exceeds MaxTbSizeY, the codec block may first be split into a default number of four sub-blocks (as shown in FIG. Figure 3 As discussed above, it is assumed that the minimum sample requirement of the four sub-blocks can be met.) If the resulting four sub-blocks have a width exceeding MaxTbSizeY, then the four sub-blocks will be split using the vertical split type to produce two or four further sub-blocks per sub-block (depending on the width of each sub-block and the minimum sample requirement), and the dimensions of these further sub-blocks can be tested to see if they meet the MaxTbSizeY size threshold. The above process of splitting sub-blocks can be processed recursively until the last sub-block meets the MaxTbSizeY size threshold.

[0191] In the illustrative example of MaxTbSizeY=32 and the codec block 402 has cbWidth 64 and cbHeight 128 (i.e., the codec block 402 is a 64×128 block), the horizontal ISP partition will produce a sub-block of size 64×32, where the width of the sub-block will exceed MaxTbSizeY. In this case, the transform_tree() syntax structure of Table 4 will be applied with the NO_ISP_SPLIT mode (this is only used for partitioning, intra prediction will follow the ISP process), and each 64×32 sub-block will be divided into two 32×32 blocks.

[0192] For the case where the split type is vertical (ISP_VER_SPLIT) or horizontal (ISP_HOR_SPLIT) (i.e., when the split type is set to a value not equal to ISP_NO_SPLIT) and one of the width or height of the coding block or sub-block exceeds MaxTbSizeY, the transform_tree() structure of Table 4 may be modified. For example, for horizontal and vertical split types, the syntax tables of coding_unit() and transform_tree() may be modified from Tables 2 and 4, respectively, where the modifications are identified in italic text in Tables 2' and 4' below, respectively.

[0193]

[0194]

[0195] Table 2'- Modified codec unit syntax table

[0196]

[0197]

[0198] Table 4' - Modified Transform Tree Syntax Table

[0199] In an alternative implementation, the split type utilized by the transform_tree() of a sub-block can be set to a value where the split type may result in a reduction in the width or height dimension that exceeds MaxTbSizeY. In such an example, the number of sub-partitions for each stage of transform tree reduction may be the same or derived separately. Consider the same example as above, where MaxTbSizeY=32, cbWidth 64 and cbHeight is 128 (i.e., 64×128 codec blocks), the horizontal ISP split will produce sub-blocks of size 64×32, where the width will exceed MaxTbSizeY. In this case, the transform_tree() syntax structure of Table 4' can be used when the split type is NO_ISP_SPLIT and each 64×32 sub-block will be split into two 32×32 blocks. For the case where the split type is horizontal or vertical, the syntax structure of transform_tree() shown in Table 4" below may be utilized in some alternative methods. Table 4" below includes modifications to Table 4 highlighted in italic text, where Table 4" may be used for implementations that set the split type to horizontal or vertical based on whether horizontal or vertical splitting, respectively, results in a reduction in a dimension (height or width) that exceeds MaxTbSizeY:

[0200]

[0201]

[0202] Table 4" - Modified Transform Tree Syntax Table

[0203] In other examples where ISP mode is enabled, for each sub-block obtained from a partitioned block using ISP mode, the transform_tree() structures of Table 4' and Table 4" can be used for either of the two partition types (horizontal partition type or vertical partition type). The modified transform_tree() can be selected between Table 4' and Table 4" based on the relationship between the width and height of the block and MaxTbSizeY.

[0204] In the above example of using the ISP mode to divide the codec block into sub-blocks, when one or more resulting sub-blocks are further divided, the value of the partition index (partIdx) of the sub-block can be assigned based on the processing order of the sub-blocks. In one example, partIdx can be assigned to the sub-blocks so that only the first sub-partition (in normal or reverse order as described above) is assigned a partIdx value of 0, and the remaining sub-partitions are assigned values ​​according to the decoding order of the sub-partitions. In other examples, one or more sub-blocks can be assigned a partIdx value of 0, indicating that the same processing is applied to all sub-blocks that are currently assigned a partIdx value of 0.

[0205] In another example of using ISP partitioning of a codec block when ISP mode is enabled, the number of sub-partitions into which the codec block is divided is allocated so that the width and height of any resulting sub-block do not exceed the value of MaxTbSizeY. The following semantics identify the division of the codec block into a number of sub-blocks to ensure that the resulting sub-blocks meet the MaxTbSizeY size threshold. Italic text identifies modifications to existing implementations of ISP partitioning.

[0206] The variable NumIntraSubPartitions specifies the number of transform block subpartitions that the intra luma codec block is divided into. NumIntraSubPartitions is derived as follows:

[0207] – If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT, NumIntraSubPartitions is set to 1.

[0208] – Otherwise, NumIntraSubPartitions is set to 2 if one of the following conditions is true:

[0209] –cbWidth is equal to 4 and cbHeight is equal to 8,

[0210] –cbWidth is equal to 8 and cbHeight is equal to 4.

[0211] – Otherwise, if cbWidth<=MaxTbSizeY, cbHeight>MaxTbSizeY and IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, then NumIntraSubPartitions is set equal to Max(4, cbHeight / MaxTbSizeY).

[0212] – Otherwise, if cbWidth>MaxTbSizeY, cbHeight<=MaxTbSizeY and IntraSubPartitionsSplitType is equal to ISP_VER_SPLIT, then NumIntraSubPartitions is set equal to Max(4, cbWidth / MaxTbSizeY).

[0213] – Otherwise, NumIntraSubPartitions is set equal to 4.

[0214] In some implementations, in addition to the above changes, the following constraints are added so that the values ​​of intra_subpartitions_mode_flag and intra_subpartitions_split_flag are constrained so that the width and height of the subpartitions generated from the split codec block are both less than or equal to MaxTbSizeY.

[0215] In some implementations of a codec block with ISP mode enabled, the codec block can be divided horizontally and vertically when the width cbWidth of the codec block is greater than MaxTbSizeY or the height cbHeight of the codec block is greater than MaxTbSizeY (or both cbWidth and cbHeight are greater than MaxTbSizeY). For this case, the number of subpartitions NumIntraSubPartitions can be assigned to be equal to 4. In some implementations, NumIntraSubPartitions can be derived to be equal to the maximum of 4 and cbWidth*cbHeight / MaxTbSizeY / MaxTbSizeY, represented by the expression Max(4, cbWidth*cbHeight / MaxTbSizeY / MaxTbSizeY). In this implementation, the order of decoding subpartitions can be defined similar to the order of decoding codec units, or defined as another scanning order that can be specified for ISP mode.

[0216] In another example of partitioning a codec block when ISP mode is enabled, a constraint may be added such that the value of intra_subpartitions_mode_flag is not allowed to be 1 when the width or height of any sub-block resulting from partitioning the codec block is larger than MaxTbSizeY. The following semantics illustrate such an implementation, with modifications to existing implementations highlighted in italic text:

[0217] The bitstream conformance requirements are:

[0218] For a codec block with width cbWidth and height cbHeight, when intra_sub_partitions_mode_flag[x0][y0] is equal to 1, the values ​​of sbWidth and sbHeight shall both be less than or equal to MaxTbSizeY, where sbWidth and sbHeight are derived as follows:

[0219] – When IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, sbWidth is set equal to the width of cbWidth and sbHeight is set equal to cbHeight / NumIntraSubPartitions.

[0220] – When IntraSubPartitionsSplitType is equal to ISP_VER_SPLIT, sbWidth is set equal to the width of cbWidth / NumIntraSubPartitions and sbHeight is set equal to cbHeight.

[0221] In another example, the signaling of the syntax element intra_subpartitions_mode_flag is constrained such that when a subblock resulting from partitioning a codec block would have a width or height (or both) greater than MaxTbSizeY, the value of intra_subpartitions_mode_flag is not signaled and is inferred to be equal to 0. For example, if any subblock obtained from partitioning a codec block may have one or more of its width and height greater than MaxTbSizeY, the ISP mode for the codec block may be disabled.

[0222] An example showing changes to the existing syntax and / or semantics of JVET WD4 for VVC with respect to VPDU boundaries is now described. Italic text shows additions to existing implementations of the specification, and strikethrough text shows text removed from the specification. In this example, a constraint may be added to ensure that a codec block encoded and decoded with ISP mode enabled does not cross a VPDU boundary. Semantics may be added to the specification as follows:

[0223] The bitstream conformance requirement is that when cbWidth or cbHeight is greater than SizeV, the value of intra_subpartitions_mode_flag shall be equal to 0.

[0224] In some cases, the width and height constraints of the VPDU may not be equal, in which case the bitstream constraints apply to the width and height of the CU separately.

[0225] In some cases, a constraint may be imposed in the syntax as follows: when cbWidth or cbHeight is greater than SizeV, the syntax element intra_subpartitions_mode_flag is not signaled and the value of intra_subpartitions_mode_flag is inferred to be equal to 0.

[0226] As previously described, a virtual pipeline data unit (VPDU) is a non-overlapping cell in a picture or video frame that can be used for parallel processing. In some examples, the VPDU structure may include a virtual block for memory access (e.g., for determining which memory area is used to process one or more specific data blocks), which defines the size of the memory allocated to implement a standard-based codec process (e.g., HEVC, VVC, or other encoding process). Although the VPDU structure may not correspond to a block partitioning mechanism for codec purposes, the VPDU can be used in a video processing pipeline. For example, in a hardware decoding process, consecutive VPDUs can be processed in parallel by multiple processing / decoding pipeline stages (e.g., different decoding pipeline stages process different VPDUs simultaneously). In some cases, in some pipelines, the VPDU size may be roughly proportional to the buffer size. For example, the VPDU size may be set to the size of a transform block (TB) size. In an illustrative example, the size of the VPDU may be 64×64 samples (e.g., luminance samples). In HEVC, the VPDU size is set to the maximum transform block size, i.e., 32×32-L (luminance samples) and 16×16-C (chrominance samples). In VVC, the VPDU size is set to 64×64-L (luminance samples) and 32×32-C (chrominance samples), which results in a request for a larger VPDU size.

[0227] In some examples, a VPDU may contain one or more blocks (e.g., CUs, PUs, TUs, or other blocks). For example, in some cases, a single CU may be included in a VPDU (e.g., the size of the CU is the same as the size of the VPDU). Figure 4 , if the size of the codec block 402 corresponds to the VPDU size, the codec block 402 can be contained in a single VPDU. In an illustrative example where the codec block 402 and the VPDU are square blocks and the size of each side of the VPDU is SizeV, the width cbWidth and height cbHeight of the codec block 402 will both be equal to SizeV. In such examples where the VPDU can completely contain the codec block, the codec block will not cross the VPDU boundary. Therefore, if the codec block 402 is encoded and decoded using the ISP mode, the codec block 402 will not violate the VPDU constraint by crossing the VPDU boundary. Therefore, in such an example, when cbWidth and cbHeight are less than or equal to the size threshold SizeV of the VPDU, the syntax element intra_subpartitions_mode_flag can be signaled with a value of 1 indicating that the ISP mode is enabled for the codec block 402.

[0228] Figure 5 5 is a diagram showing an example of multiple codec units of a VPDU 500. In one such example, each of the multiple CUs contained in the VPDU 500 may have a size that is smaller than the VPDU size. As shown, four blocks or codec units are identified in the VPDU 500, including a first block 520, a second block 522, a third block 524, and a fourth block 526. Each of the first block 520, the second block 522, the third block 524, and / or the fourth block 526 may be a CU, PU, ​​TU, or other block of a picture. The dimensions (width cbWidth and height cbHeight) have been identified for the first block 520 to illustrate the relationship between these dimensions and the size of the VPDU 500. Accordingly, the height SizeV_height and the width SizeV_width have also been identified for the VPDU 500, where in some examples, SizeV_height and SizeV_width may be equal to the same value SizeV.

[0229] In the above example, the width cbWidth of the first block 520 will be less than the width SizeV_width of the VPDU 500, and the height cbHeight of the first block 520 will be less than the height SizeV_height of the VPDU 500. Therefore, if the first block 520 is encoded and decoded with the ISP mode, the first block 520 will not violate the VPDU constraint by crossing the VPDU boundary. Therefore, in such an example, when cbWidth and cbHeight are less than or equal to the size threshold corresponding to the VPDU boundary, the ISP mode can be allowed for the first block 520, and the syntax element intra_subpartitions_mode_flag can be signaled with a value of 1 indicating that the ISP mode is used for the first block 520. Although not explicitly shown, the dimensions of the second block 522, the third block 524, and the fourth block 526 can be the same as the dimensions discussed above with reference to the first block 520. Therefore, in such an example, the second block 522, the third block 524, and the fourth block 526 can also be encoded and decoded with the enabled ISP mode. In some examples, as explained previously and will be further described below, when ISP mode is allowed based on VPDU constraints, whether ISP mode can be enabled or used can be based on a further comparison of cbWidth and cbHeight with a maximum transform block size threshold MaxTbSizeY.

[0230] In some examples, the size of a codec block (e.g., a CU, PU, ​​or other block) can span multiple VPDUs. For example, multiple VPDUs can be used to process a codec block. For example, a codec block of size 128×64 (128 samples wide × 64 samples high) or a codec block of size 64×128 (64 samples wide × 128 samples high) can span two VPDUs, each of which is 64×64 in size. In another example, a codec block of size 128×128 (128 samples wide × 128 samples high) can span four VPDUs, each of which is 64×64 in size. The codec block can be partitioned into a certain number of sub-blocks for inter-frame prediction to be performed by each VPDU pipeline stage. For example, a 128×128 codec block can be partitioned into 64×64 sub-blocks for processing by four different VPDU pipeline stages. Since inter-frame prediction is not performed relying on neighboring blocks, the codec block can be partitioned for inter-frame prediction. However, such codec blocks spanning multiple VPDUs also cross VPDU boundaries, which may violate the VPDU constraint regarding allowing the use of intra-frame sub-partitioning mode to split the codec blocks.

[0231] Figure 6602. The codec block 602 may include samples associated with four VPDUs, including a first VPDU 630, a second VPDU 632, a third VPDU 634, and a fourth VPDU 636. The codec block 602 may have dimensions including a width cbWidth and a height cbHeight. Each of the four VPDUs may have dimensions including a width and a height of SizeV (as shown in the representative illustration for the first VPDU 630). In an illustrative example, the current block 602 has a size of 128×128, and each of the VPDUs 630-634 has a size of 64×64. The codec block 602 may be a CU, PU, ​​TU, or other block of a picture.

[0232] exist Figure 6 In the example of , both cbWidth and cbHeight of codec block 602 are greater than SizeV of each of the four VPDUs 630-634. Therefore, the dimensions of codec block 602 will violate the requirement to encode codec block 602 in ISP mode. As previously described, in example implementations, bitstream conformance may include the requirement that when cbWidth or cbHeight of a codec block is greater than SizeV, the value of intra_subpartitions_mode_flag shall be equal to 0. In examples where the width and height of a VPDU may not be equal (e.g., as shown in reference Figure 6 ), bitstream conformance may include a requirement that when cbWidth is greater than SizeV_width or cbHeight is greater than SizeV_height, the value of intra_subpartitions_mode_flag should be equal to 0 to indicate that ISP mode is not enabled for the codec block 602. In some examples, based on the cbWidth and cbHeight of the codec block 602 being greater than SizeV of the first VPDU 630 (and similarly, greater than SizeV of the remaining VPDUs 632, 634, and 636), the intra_subpartitions_mode_flag of the codec block 602 may be set to 0.

[0233] In another alternative as previously described, a constraint may be imposed in the syntax for the ISP mode: when cbWidth or cbHeight is greater than SizeV, the syntax element intra_subpartitions_mode_flag is not signaled, and the value of intra_subpartitions_mode_flag is inferred to be equal to 0. Thus, in the case of the codec block 602, since cbWidth and cbHeight are greater than SizeV of one or more VPDUs including four VPDUs 630-636, intra_subpartitions_mode_flag is not signaled (e.g., by an encoding device) for the codec block 602. In such a case where the intra_subpartitions_mode_flag is not signaled or included in the coded bitstream in which the codec block 602 is included, the value of intra_subpartitions_mode_flag may be inferred to be equal to 0 (e.g., by a decoding device receiving the coded bitstream).

[0234] In some examples, determining whether ISP mode is allowed based on the above comparison of the dimensions of the codec block to the VPDU boundaries can be combined with other considerations for enabling ISP mode when ISP mode is allowed. For example, when one or more VPDU constraints are met, intra_subpartitions_mode_flag can be signaled to indicate that ISP mode is enabled for the block. For example, as discussed above with reference to existing implementations, when one of cbWidth or cbHeight is greater than MaxTbSizeY and the other of cbWidth or cbHeight is less than MaxTbSizeY, or if both cbWidth and cbHeight are less than MaxTbSizeY, then intra_subpartitions_mode_flag can be signaled for the codec block (e.g., to indicate that ISP is enabled).

[0235] Therefore, in the example where the maximum transform block size MaxTbSizeY is the same as the VPDU, if the cbWidth or cbHeight of the codec block is greater than SizeV (where SizeV is equal to MaxTbSizeY), then ISP mode is not allowed and therefore intra_subpartitions_mode_flag is not signaled for the codec block because the dimensions of the codec block would violate one or more VPDU constraints by crossing a VPDU boundary. For example, if in the illustrative example, SizeV and MaxTbSizeY are both equal to 64, then for either cbWidth or cbHeight greater than 64, the VPDU constraints will prevent ISP mode from being allowed, and intra_subpartitions_mode_flag is not signaled for the codec block (i.e., ISP mode is not enabled).

[0236] However, if in another illustrative example, MaxTbSizeY is 32 and SizeV is 64, then the intra_subpartitions_mode_flag may be allowed and enabled in some cases. For example, if the cbWidth of the codec block is equal to 64 and the cbHeight is equal to 32, then in this example, cbWidth and cbHeight will be less than SizeV, resulting in a determination that ISP mode is allowed based on satisfying the VPDU constraint. In addition, since cbWidth equal to 64 will cause cbWidth to be greater than MaxTbSizeY of size 32, and cbHeight equal to 32 will cause cbHeight to be less than or equal to MaxTbSizeY of size 32, ISP mode for the codec block can be enabled and intra_subpartitions_mode_flag can be signaled in the bitstream (e.g., by an encoding device). However, since cbWidth is greater than MaxTbSizeY 32, the intra_subpartitions_split_flag is not signaled. In such an example, as explained with reference to Table 2, existing implementations of a decoding device receiving the bitstream may infer that the value of intra_subpartitions_split_flag is 0 (corresponding to horizontal split ISP_HOR_SPLIT), leading to undesirable results. Figure 7As discussed, by implementing a modification that infers the value of intra_subpartitions_split_flag as the vertical split ISP_VER_SPLIT, the desired split of a codec block with cbWidth equal to 64 and cbHeight equal to 32 will produce two sub-blocks with a width and height of 32, respectively, where these resulting sub-blocks will be constrained to be within MaxTbSizeY of size 32.

[0237] In another example, the signaling of intra_subpartitions_mode_flag may also be modified from existing implementations to disable ISP mode (or not signal intra_subpartitions_mode_flag) when the cbWidth or cbHeight of the codec block is greater than MaxTbSizeY. As previously described, when the width or height of the luma codec block is greater than the maximum transform size MaxTbSizeY, the codec block is divided into two or more transform units as specified in the transform_tree() syntax structure of Table 2. Since the syntax of intra_subpartitions_mode_flag in the existing implementation allows intra_subpartitions_mode_flag to be signaled when one of the cbWidth and cbHeight of the codec block is greater than MaxTbSizeY and the other of cbWidth and cbHeight is less than or equal to MaxTbSizeY, the above-mentioned problem of inferring intra_subpartitions_split_flag to be 0 (corresponding to horizontal splitting ISP_HOR_SPLIT) regardless of whether vertical splitting or horizontal splitting is the best choice occurs. However, if intra_subpartitions_mode_flag is signaled only when both cbWidth and cbHeight are less than or equal to MaxTbSizeY, the syntax of both intra_subpartitions_mode_flag and intra_subpartitions_split_flag will become consistent with each other. For example, Table 2" shown below illustrates in italic text the modification to the signaling of intra_subpartitions_mode_flag when cbWidth<=MaxTbSizeY&&cbHeight<=MaxTbSizeY (where, in the existing implementation of Table 2, intra_subpartitions_mode_flag is signaled when (cbWidth<=MaxTbSizeY||cbHeight<=MaxTbSizeY)).

[0238]

[0239] Table 2" - Modified Transform Tree Syntax Table

[0240] In an implementation where intra_subpartitions_mode_flag is signaled for a codec block when cbWidth is less than or equal to a size threshold MaxTbSizeY and cbHeight is less than or equal to a size threshold MaxTbSizeY, it is understood that the signaling of intra_subpartitions_mode_flag based on the relationship between the dimensions of the codec block and the size threshold is the same when the size threshold is based on the VPDU size (SizeV) or the maximum transform block size (MaxTbSizeY). Thus, when both cbWidth and cbHeight are less than the size threshold (where the size threshold is SizeV equal to MaxTbSizeY), intra_subpartitions_mode_flag is signaled to enable ISP mode for a codec block that allows ISP mode. If either cbWidth or cbHeight violates the size threshold, intra_subpartitions_mode_flag is not signaled, and ISP mode may not be allowed or disabled for the codec block.

[0241] Figure 7 7 is a flow chart illustrating an example of a process 700 for decoding video data by determining a sub-partition type to be applied to a codec block with intra_subpartition (ISP) mode enabled. At 702, the process 700 includes obtaining an encoded video bitstream including video data. In some examples, the process of obtaining the encoded video bitstream can be performed by a decoding device. As described above, the video data can include multiple pictures, and the pictures can be divided into multiple blocks. The video data can also include motion information of the pictures and / or blocks that can be used to perform motion compensation.

[0242] At 704, the process 700 includes determining that there is no intra-subpartition split flag for the current block of video data in the video bitstream, the intra-subpartition split flag specifying whether the split type of the intra-subpartition mode for the current block is horizontal or vertical. For example, when cbWidth or cbHeight is less than or equal to the size threshold MaxTbSizeY or if both cbWidth and cbHeight are less than or equal to MaxTbSizeY, the ISP mode flag intra_subpartitions_mode_flag of the codec block 402 may have been signaled. Accordingly, in examples where one of cbWidth or cbHeight is greater than MaxTbSizeY and the other is less than MaxTbSizeY, intra_subpartitions_mode_flag will be signaled as true (with a value indicating that ISP mode is enabled for the codec block). In such an implementation, when one of cbWidth or cbHeight is greater than MaxTbSizeY, the intra_subpartitions_split_flag may not be present in the signaled bitstream.

[0243] At 706 , process 700 includes determining that at least one of the width and height of the current block is greater than a size threshold corresponding to the maximum transform block size. For example, when intra_subpartitions_split_flag is not present for codec block 402 , cbWidth or cbHeight may be determined to be greater than MaxTbSizeY.

[0244] At 708, process 700 includes determining a value of an intra subpartition split flag for the current block based on whether the width or height of the current block is greater than a size threshold. For example, if cbWidth is greater than MaxTbSizeY, it can be inferred that for the codec block, intra_subpartitions_split_flag is equal to a first value (e.g., a value of 1 corresponding to ISP vertical split ISP_VER_SPLIT). On the other hand, if cbHeight is greater than MaxTbSizeY, it can be inferred that for the codec block, intra_subpartitions_split_flag is equal to a second value (e.g., a value of 0 corresponding to ISP horizontal split ISP_HOR_SPLIT).

[0245] Figure 88 is a flow chart illustrating an example of a process 800 for decoding video data by defining a partitioning structure of an ISP mode to ensure that the dimensions of sub-blocks obtained from a segmented codec block meet a size threshold. At 802, the process 800 includes obtaining an encoded video bitstream including video data. In some examples, the process of obtaining the encoded video bitstream can be performed by a decoding device. As described above, the video data can include multiple pictures, and the pictures can be divided into multiple blocks. The video data can also include motion information of the pictures and / or blocks that can be used to perform motion compensation.

[0246] At 804, process 800 includes determining that an intra-subpartition mode of the current block for dividing video data is enabled for the current block. For example, when cbWidth or cbHeight is less than or equal to a size threshold MaxTbSizeY or if both cbWidth and cbHeight are less than or equal to MaxTbSizeY, an ISP mode flag intra_subpartitions_mode_flag of the codec block 402 may be signaled. Accordingly, in examples where one of cbWidth or cbHeight is greater than MaxTbSizeY and the other is less than MaxTbSizeY, intra_subpartitions_mode_flag is signaled with a value (e.g., a value of 1) indicating that the ISP mode is enabled for the codec block. In some examples, if a dimension (width or height) of a sub-block resulting from partitioning the codec block into sub-blocks using the ISP mode is greater than the size threshold MaxTbSizeY, intra_subpartitions_mode_flag may not be signaled.

[0247] At 806, process 800 includes determining a partitioning structure for partitioning the current block into sub-blocks, wherein the dimensions of the sub-blocks resulting from the partitioning do not exceed a size threshold. For example, in some implementations, if the dimensions of the sub-blocks exceed the size threshold, the sub-blocks obtained from the partition codec block 402 may be partitioned using a recursive structure according to the modified transform_tree() syntax structure shown in Table 4' or Table 4". For example, if the width or height of the sub-block exceeds MaxTbSizeY, the resulting sub-blocks may be recursively partitioned using a vertical or horizontal partitioning type until a sub-block whose dimensions do not exceed the MaxTbSizeY size threshold is obtained.

[0248] Fig. 9900 is a flow chart illustrating an example of a process 900 for encoding video data by determining whether to enable ISP mode for a codec block based on whether the codec block violates one or more constraints (e.g., crosses a VPDU boundary). At 902, process 900 includes obtaining a current block of a picture of video data. In some examples, such as when process 900 is performed by an encoding device, the video data may include unencoded video data. As described above, the video data may include multiple pictures, and the pictures may be divided into multiple blocks. Process 900 may determine motion information of the picture and / or block that may be used to perform motion compensation.

[0249] At 904, process 900 includes determining whether a width of a current block of the picture is greater than at least one of a width size threshold and a height of the current block is greater than a height size threshold, the width size threshold corresponding to the width of the data unit and the height size threshold corresponding to the height of the data unit. In some cases, the data unit is a virtual pipeline data unit (VPDU). In some cases, the data unit is a transform block. In the case where the data unit is a transform block, the width size threshold and the height size threshold can be defined to be equal to a maximum transform block size (e.g., MaxTbSizeY). In one illustrative example, reference Figure 5 , when the data unit is a VPDU, the cbWidth and cbHeight of the first block 520 may be compared with the SizeV_width and SizeV_Height of the VPDU 500, respectively, to determine whether the cbWidth is greater than the SizeV_width or whether the cbHeight is greater than at least one of the SizeV_height. In another example, referring to Figure 6 , cbWidth and cbHeight of the codec block 602 may be compared with SizeV (for the width dimension) and SizeV (for the height dimension) of the VPDU 630 (where the width dimension threshold is equal to the height dimension threshold, which is equal to SizeV), respectively, to determine whether cbWidth is greater than SizeV or whether cbHeight is greater than SizeV.

[0250] At 906, process 900 includes determining whether to signal an intra sub-partition mode flag for the current block based on at least one of whether a width of the current block is greater than a width size threshold and whether a height of the current block is greater than a height size threshold, wherein a value of the intra sub-partition mode flag indicates whether an ISP mode is applied to the current block to divide the current block into rectangular transform block sub-partitions. Figure 6In the example of , both cbWidth and cbHeight of the codec block 602 are greater than SizeV of each of the four VPDUs 630-634. Therefore, the dimensions of the codec block 602 will violate one or more VPDU constraints for the codec block 602 encoded and decoded with the ISP mode. As previously described, the bitstream consistency may include the following requirement: when the cbWidth or cbHeight of the codec block is greater than SizeV, the value of intra_subpartitions_mode_flag should be equal to 0. In such an example, when the width of the current block is greater than the width size threshold or the height of the current block is greater than the height size threshold, the intra-frame subpartition mode is not allowed for the current block. In such an example, when the intra-frame subpartition mode is not allowed for the codec block, the intra_subpartition_mode_flag is not signaled in the encoded video bitstream from the encoding device, and the decoding device can infer that the value of the intra_subpartition_mode_flag is a first value (0), which indicates that the intra-frame subpartition mode is not allowed for the codec block.

[0251] At block 908, process 900 includes generating a coded video bitstream including at least the current block based on determining whether to signal an intra subpartition mode flag for the current block. For example, the coded bitstream may include the current block and an intra_subpartitions_mode_flag indicating that the ISP mode is enabled for the current block when the current block does not violate the VPDU constraint.

[0252] exist Figure 5 In the example, when both cbWidth and cbHeight are less than or equal to the size threshold corresponding to the VPDU boundary, the syntax element intra_subpartitions_mode_flag can be signaled with a value indicating whether the ISP mode is enabled for the first block 520 (e.g., a value of 1 or "true" indicates that the ISP mode is enabled for the first block 520, or a value of 0 or "false" indicates that the ISP mode is not performed on the first block 520). In some examples, based on determining that the intra-frame sub-partition mode is enabled, the process 700 can be used to determine the value of the intra-frame sub-partition split flag when the intra-frame sub-partition split flag is not signaled.

[0253] In some implementations, the processes (or methods) described herein, including processes 700, 800, and 900, may be performed by a computing device or apparatus (e.g., Figure 1 For example, it can be performed by the system 100 shown in FIG. Figure 1 and Fig.10 The encoding device 104, another video source side device or video transmission device, Figure 1and Fig.11 The process may be performed by a decoding device 112 and / or another client-side device (e.g., a player device, a display, or any other client-side device) as shown. In some examples, the computing device or apparatus may include a camera configured to capture video data (e.g., a video sequence) including video frames. In some examples, the camera or other capture device that captures the video data is separate from the computing device, in which case the computing device receives or obtains the captured video data.

[0254] In some cases, the computing device or apparatus may include one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, and / or other (multiple) components configured to perform the steps of the processes described herein. In some examples, the computing device may include a mobile device, a desktop computer, a server computer, and / or a server system, or other types of computing devices. The computing device may also include a network interface configured to transmit video data. The network interface may be configured to transmit data based on an Internet Protocol (IP) or other types of data. In some examples, the computing device or apparatus may include a display for displaying output video content (e.g., samples of a picture of a video bitstream).

[0255] The components of a computing device (e.g., one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, and / or other components) may be implemented in circuits. For example, the components may include and / or may be implemented using electronic circuits or other electronic hardware, which may include one or more programmable electronic circuits (e.g., microprocessors, graphics processing units (GPUs), digital signal processors (DSPs), central processing units (CPUs), and / or other suitable electronic circuits), and / or may include and / or be implemented using computer software, firmware, or any combination thereof to perform the various operations described herein.

[0256] The processes 700, 800, 900 are illustrated as logical flow diagrams, the operations of which represent a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the operations. Typically, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement the process.

[0257] In addition, the processes described herein, including processes 800 and 900, may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed together on one or more processors by hardware or a combination thereof. As described above, the code may be stored on a computer-readable or machine-readable storage medium, for example, in the form of a computer program including multiple instructions that may be executed by one or more processors. The computer-readable or machine-readable storage medium may be non-transitory.

[0258] The coding and decoding techniques discussed herein can be implemented in an example video encoding and decoding system (e.g., system 100). In some examples, the system includes a source device that provides encoded video data that will be decoded by a destination device later. In particular, the source device provides video data to the destination device via a computer-readable medium. The source device and the destination device may include any of a variety of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" mobile phones, so-called "smart" flat panels, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, the source device and the destination device may be equipped for wireless communication.

[0259] The destination device can receive the encoded video data to be decoded via a computer-readable medium. The computer-readable medium can include any type of medium or device capable of moving the encoded video data from the source device to the destination device. In one example, the computer-readable medium can include a communication medium so that the source device can directly send the encoded video data to the destination device in real time. The encoded video data can be modulated according to a communication standard such as a wireless communication protocol and sent to the destination device. The communication medium can include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form a part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium can include a router, a switch, a base station, or any other equipment that can be used to facilitate communication from the source device to the destination device.

[0260] In some examples, the encoded data can be output from the output interface to the storage device. Similarly, the encoded data can be accessed from the storage device through the input interface. The storage device may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, a flash memory, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. In another example, the storage device may correspond to a file server or another intermediate storage device that can store the encoded video generated by the source device. The destination device can access the stored video data from the storage device via streaming or downloading. The file server can be any type of server that can store encoded video data and send the encoded video data to the destination device. Example file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The destination device can access the encoded video data through any standard data connection (including an Internet connection). This can include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a DSL, a cable modem, etc.) suitable for accessing the encoded video data stored on the file server, or a combination of both. The transmission of the encoded video data from the storage device can be streaming, downloading, or a combination thereof.

[0261] The technology of the present disclosure is not necessarily limited to wireless applications or settings. The technology can be applied to video encoding and decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (e.g., dynamic adaptive streaming over HTTP (DASH)), digital video encoded on a data storage medium, decoding of digital video stored on a data storage medium, or other applications. In some examples, the system can be configured to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and / or video telephony.

[0262] In one example, the source device includes a video source, a video encoder and an output interface. The destination device may include an input interface, a video decoder and a display device. The video encoder of the source device may be configured to apply the technology disclosed herein. In other examples, the source device and the destination device may include other components or arrangements. For example, the source device may receive video data from an external video source (such as an external camera). Similarly, the destination device may interface with an external display device, rather than including an integrated display device.

[0263] The example system above is just an example. The techniques for parallel processing of video data can be performed by any digital video encoding and / or decoding device. Although the techniques of the present disclosure are generally performed by video encoding devices, these techniques can also be performed by video encoders / decoders (commonly referred to as "codecs"). In addition, the techniques of the present disclosure can also be performed by a video preprocessor. The source device and the destination device are merely examples of such codec devices, in which the coded and decoded video data is generated for transmission to the destination device. In some examples, the source device and the destination device can operate in a substantially symmetrical manner so that each of the devices includes a video encoding and decoding component. Therefore, the example system can support one-way or two-way video transmission between video devices, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0264] The video source can include a video capture device, such as a video camera, a video archive comprising previously captured video, and / or a video feed interface receiving video from a video content provider. As another alternative, the video source can generate computer graphics-based data as the source video, or generate a combination of real-time video, archived video, and computer-generated video. In some cases, if the video source is a video camera, the source device and the destination device can form a so-called camera phone or video phone. However, as mentioned above, the technology described in the disclosure can be applied to video coding and decoding generally, and can be applied to wireless and / or wired applications. In each case, the captured, pre-captured or computer-generated video can be encoded by a video encoder. The video information encoded can then be output to a computer-readable medium by an output interface.

[0265] As described above, computer-readable media may include transient media such as wireless broadcast or wired network transmissions, or storage media such as hard disks, flash drives, compact discs, digital video discs, Blu-ray discs, or other computer-readable media (i.e., non-transient storage media). In some examples, a network server (not shown) may receive encoded video data from a source device and provide the encoded video data to a destination device, for example, via a network transmission. Similarly, a computing device of a media production facility such as a disc stamping device may receive encoded video data from a source device and generate a disc containing the encoded video data. Therefore, in various examples, computer-readable media may be understood to include one or more computer-readable media in various forms.

[0266] The input interface of the destination device receives information from the computer-readable medium. The information of the computer-readable medium may include syntax information defined by the video encoder, which is also used by the video decoder, including syntax elements that describe the characteristics and / or processing of blocks and other coding and decoding units (e.g., groups of pictures (GOPs)). The display device displays the decoded video data to the user and may include any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device. Various embodiments of the present application have been described.

[0267] The details of the encoding device 104 and the decoding device 112 are respectively described in Fig.10 and Fig.11 Shown in. Fig.10 1 is a block diagram illustrating an example encoding device 104 that can implement one or more of the techniques described in the present disclosure. The encoding device 104 can, for example, generate the syntax structures described herein (e.g., syntax structures of VPS, SPS, PPS or other syntax elements). The encoding device 104 can perform intra-frame prediction and inter-frame prediction coding and decoding of video blocks within a video slice. As previously described, intra-frame coding and decoding relies at least in part on spatial domain prediction to reduce or eliminate spatial domain redundancy within a given video frame or picture. Inter-frame coding and decoding relies at least in part on temporal domain prediction to reduce or eliminate temporal domain redundancy within adjacent or surrounding frames of a video sequence. Intra-frame mode (I mode) can refer to any of several spatial domain-based compression modes. Inter-frame modes such as unidirectional prediction (P mode) or bidirectional prediction (B mode) can refer to any of several temporal domain-based compression modes.

[0268] The encoding device 104 includes a partition unit 35, a prediction processing unit 41, a filter unit 63, a picture memory 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy encoding unit 56. The prediction processing unit 41 includes a motion estimation unit 42, a motion compensation unit 44, and an intra-frame prediction processing unit 46. For video block reconstruction, the encoding device 104 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62. The filter unit 63 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although the filter unit 63 is not limited to a single-loop filter, the filter unit 63 is not limited to a single-loop filter. Fig.10 6 is shown as an in-loop filter, but in other configurations, filter unit 63 may be implemented as a post-loop filter. Post-processing device 57 may perform additional processing on the encoded video data generated by encoding device 104. The techniques of this disclosure may be implemented by encoding device 104 in some cases. However, in other cases, one or more of the techniques of this disclosure may be implemented by post-processing device 57.

[0269] like Fig.10 As shown, the encoding device 104 receives video data, and the partitioning unit 35 partitions the data into video blocks. The partitioning may also include partitioning into slices, slice segments, tiles, or other larger units, and video block partitioning according to, for example, a quadtree structure of LCUs and CUs. The encoding device 104 generally shows a component for encoding a video block within a video slice to be encoded. The slice may be divided into a plurality of video blocks (and may be divided into a set of video blocks referred to as tiles). The prediction processing unit 41 may select one of a plurality of possible codec modes for the current video block based on error results (e.g., codec rate and distortion level, etc.), such as one of one or more inter-frame prediction codec modes in a plurality of intra-frame prediction codec modes. The prediction processing unit 41 may provide the resulting intra-frame or inter-frame codec block to the adder 50 to generate residual block data, and to the adder 62 to reconstruct the coding block used as a reference picture.

[0270] Intra-prediction processing unit 46 within prediction processing unit 41 may perform intra-prediction encoding and decoding of the current video block relative to one or more neighboring blocks in the same frame or slice as the current block to be encoded to provide spatial compression. Motion estimation unit 42 and motion compensation unit 44 within prediction processing unit 41 may perform inter-prediction encoding and decoding of the current video block relative to one or more prediction blocks in one or more reference pictures to provide temporal compression.

[0271] Motion estimation unit 42 may be configured to determine an inter-prediction mode for a video slice according to a predetermined pattern for a video sequence. The predetermined pattern may specify a video slice in a sequence as a P slice, a B slice, or a GPB slice. Motion estimation unit 42 and motion compensation unit 44 may be highly integrated, but are shown separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is the process of generating motion vectors, which estimate the motion of a video block. For example, a motion vector may indicate the displacement of a prediction unit (PU) of a video block within a current video frame or picture relative to a prediction block within a reference picture.

[0272] A prediction block is a block that is found to closely match a PU of a video block to be encoded and decoded in terms of pixel difference, which may be determined by a sum of absolute difference (SAD), a sum of squared difference (SSD), or other difference metrics. In some examples, encoding device 104 may calculate values ​​for sub-integer pixel positions of a reference picture stored in picture memory 64. For example, encoding device 104 may interpolate values ​​for quarter-pixel positions, eighth-pixel positions, or other fractional pixel positions of a reference picture. Thus, motion estimation unit 42 may perform motion searches relative to full pixel positions and fractional pixel positions and output motion vectors with fractional pixel precision.

[0273] Motion estimation unit 42 calculates a motion vector for a PU of a video block in an inter-coded slice by comparing the position of the PU to the position of a prediction block of a reference picture. The reference picture may be selected from a first reference picture list (list 0) or a second reference picture list (list 1), each of which identifies one or more reference pictures stored in picture memory 64. Motion estimation unit 42 sends the calculated motion vector to entropy encoding unit 56 and motion compensation unit 44.

[0274] Motion compensation performed by the motion compensation unit 44 may involve obtaining or generating a prediction block based on a motion vector determined by motion estimation, possibly performing interpolation with sub-pixel precision. Upon receiving a motion vector for a PU of a current video block, the motion compensation unit 44 may locate the prediction block to which the motion vector points in a reference picture list. The encoding device 104 forms a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being encoded and decoded, forming pixel difference values. The pixel difference values ​​form residual data for the block and may include luminance and chrominance difference components. The adder 50 represents one or more components that perform the subtraction operation. The motion compensation unit 44 may also generate syntax elements associated with the video block and the video slice for use by the decoding device 112 when decoding the video block of the video slice.

[0275] As described above, the intra-prediction processing unit 46 may perform intra-prediction on the current block as an alternative to the inter-prediction performed by the motion estimation unit 42 and the motion compensation unit 44. Specifically, the intra-prediction processing unit 46 may determine an intra-prediction mode for encoding the current block. In some examples, the intra-prediction processing unit 46 may encode the current block using various intra-prediction modes (e.g., during separate encoding passes), and the intra-prediction processing unit 46 may select a suitable intra-prediction mode to use from the tested modes. For example, the intra-prediction processing unit 46 may calculate rate-distortion values ​​using rate-distortion analysis for various tested intra-prediction modes, and may select an intra-prediction mode with the best rate-distortion characteristics from the tested modes. The rate-distortion analysis typically determines the amount of distortion (or error) between the coded block and the original, uncoded block that is coded to produce the coded block, as well as the bit rate (i.e., the number of bits) used to produce the coded block. Intra-prediction processing unit 46 may calculate ratios from the distortions and rates for the various encoded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block.

[0276] In any case, after selecting an intra-prediction mode for a block, intra-prediction processing unit 46 may provide information indicating the selected intra-prediction mode for the block to entropy encoding unit 56. Entropy encoding unit 56 may encode the information indicating the selected intra-prediction mode. Encoding device 104 may include in the transmitted bitstream configuration data the definition of coding contexts for various blocks and an indication of the most probable intra-prediction mode for each context, an intra-prediction mode index table, and a modified intra-prediction mode index table. The bitstream configuration data may include a plurality of intra-prediction mode index tables and a plurality of modified intra-prediction mode index tables (also referred to as codeword mapping tables).

[0277] After the prediction processing unit 41 generates a prediction block for the current video block via inter-frame prediction or intra-frame prediction, the encoding device 104 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be included in one or more TUs and applied to the transform processing unit 52. The transform processing unit 52 transforms the residual video data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. The transform processing unit 52 may convert the residual video data from a pixel domain to a transform domain, such as a frequency domain.

[0278] Transform processing unit 52 may send the resulting transform coefficients to quantization unit 54. Quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization process may reduce the bit depth associated with some or all of the coefficients. The degree of quantization may be modified by adjusting a quantization parameter. In some examples, quantization unit 54 may then perform a scan of the matrix including the quantized transform coefficients. Alternatively, entropy encoding unit 56 may perform the scan.

[0279] After quantization, the entropy coding unit 56 performs entropy coding on the quantized transform coefficients. For example, the entropy coding unit 56 may perform context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) coding, or another entropy coding technique. After entropy coding by the entropy coding unit 56, the encoded bitstream may be sent to the decoding device 112, or archived for later transmission or retrieval by the decoding device 112. The entropy coding unit 56 may also entropy encode the motion vectors and other syntax elements for the current video slice being coded.

[0280] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block in the pixel domain for later use as a reference block of a reference picture. Motion compensation unit 44 may calculate a reference block by adding the residual block to a prediction block of one of the reference pictures in a reference picture list. Motion compensation unit 44 may also apply one or more interpolation filters to the reconstructed residual block to calculate sub-integer pixel values ​​for motion estimation. Adder 62 adds the reconstructed residual block to the motion compensated prediction block produced by motion compensation unit 44 to produce a reference block for storage in picture memory 64. The reference block may be used by motion estimation unit 42 and motion compensation unit 44 as a reference block to inter-predict a block in a subsequent video frame or picture.

[0281] In this way, Fig.10 The encoding device 104 represents a device configured to perform any of the techniques described herein (including the above Figure 7 , Figure 8 and / or Fig. 9 In some cases, some techniques of the present disclosure may also be implemented by post-processing device 57.

[0282] Fig.11 1 is a block diagram illustrating an example decoding device 112. The decoding device 112 includes an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, a filter unit 91, and a picture memory 92. The prediction processing unit 81 includes a motion compensation unit 82 and an intra-frame prediction processing unit 84. In some examples, the decoding device 112 may perform operations generally associated with the processing of the image from Fig.11 The decoding pass of the encoding device 104 is the reverse of the encoding process described.

[0283] During the decoding process, the decoding device 112 receives a coded video bitstream representing video blocks and associated syntax elements of a coded video slice sent by the encoding device 104. In some embodiments, the decoding device 112 may receive the coded video bitstream from the encoding device 104. In some embodiments, the decoding device 112 may receive the coded video bitstream from the encoding device 104. In some embodiments, the decoding device 112 may receive the coded video bitstream from a network entity 79 (e.g., a server, a media-aware network element (MANE), a video editor / splicer, or other such device configured to implement one or more of the above-mentioned techniques). The network entity 79 may or may not include the encoding device 104. Before the network entity 79 sends the coded video bitstream to the decoding device 112, some of the techniques described in the present disclosure may be implemented by the network entity 79. In some video decoding systems, the network entity 79 and the decoding device 112 may be part of separate devices, while in other cases, the functions described with respect to the network entity 79 may be performed by the same device that includes the decoding device 112.

[0284] The entropy decoding unit 80 of the decoding device 112 entropy decodes the bitstream to generate quantized coefficients, motion vectors and other syntax elements. The entropy decoding unit 80 forwards the motion vectors and other syntax elements to the prediction processing unit 81. The decoding device 112 can receive syntax elements at the video slice level and / or the video block level. The entropy decoding unit 80 can process and parse both fixed length syntax elements and variable length syntax elements in one or more parameter sets such as VPS, SPS and PPS.

[0285] When the video slice is coded as an intra-frame codec (I) slice, the intra-frame prediction processing unit 84 of the prediction processing unit 81 can generate prediction data for the video block of the current video slice based on the signaled intra-frame prediction mode and the data from the previously decoded block of the current frame or picture. When the video frame is coded as an inter-frame codec (i.e., B, P, or GPB) slice, the motion compensation unit 82 of the prediction processing unit 81 generates a prediction block for the video block of the current video slice based on the motion vector and other syntax elements received from the entropy decoding unit 80. The prediction block can be generated from one of the reference pictures in the reference picture list. The decoding device 112 can construct the reference frame lists (list 0 and list 1) using a default construction technique based on the reference pictures stored in the picture memory 92.

[0286] Motion compensation unit 82 determines prediction information for video blocks of the current video slice by parsing motion vectors and other syntax elements, and uses the prediction information to generate prediction blocks for the current video block being decoded. For example, motion compensation unit 82 may use one or more syntax elements in a parameter set to determine a prediction mode (e.g., intra or inter prediction) for encoding or decoding video blocks of a video slice, an inter prediction slice type (e.g., a B slice, a P slice, or a GPB slice), construction information for one or more reference picture lists for the slice, motion vectors for each inter-coded video block of the slice, inter prediction status for each inter-coded video block of the slice, and other information for decoding video blocks in the current video slice.

[0287] The motion compensation unit 82 may also perform interpolation based on an interpolation filter. The motion compensation unit 82 may calculate interpolated values ​​for sub-integer pixels of the reference block using an interpolation filter as used by the encoding device 104 during encoding of the video block. In this case, the motion compensation unit 82 may determine the interpolation filter used by the encoding device 104 from the received syntax elements and may use the interpolation filter to produce a prediction block.

[0288] The inverse quantization unit 86 inverse quantizes or dequantizes the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 80. The inverse quantization process may include determining a degree of quantization using a quantization parameter calculated by the encoding device 104 for each video block in the video slice, and likewise determining a degree of inverse quantization that should be applied. The inverse transform processing unit 88 applies an inverse transform (e.g., an inverse DCT or other suitable inverse transform), an inverse integer transform, or a conceptually similar inverse transform process to the transform coefficients to produce a residual block in the pixel domain.

[0289] After the motion compensation unit 82 generates a prediction block for the current video block based on the motion vector and other syntax elements, the decoding device 112 forms a decoded video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82. Adder 90 represents one or more components that perform this summation operation. If desired, a loop filter (in the codec loop or after the codec loop) may also be used to smooth pixel transitions or otherwise improve video quality. Filter unit 91 is intended to represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 91 is Fig.1191 is shown as an in-loop filter, but in other configurations, filter unit 91 may be implemented as a post-loop filter. The decoded video blocks in a given frame or picture are then stored in picture memory 92, which stores reference pictures for subsequent motion compensation. Picture memory 92 also stores the decoded video for later display on a display device (e.g. Figure 1 The video shown is presented on the destination device 122).

[0290] In this way, Fig.11 The decoding device 112 represents a device configured to perform any of the techniques described herein (including the above Figure 8 and / or Fig. 9 An example of a video decoder of the process described).

[0291] As used herein, the term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying (multiple) instructions and / or data. Computer-readable media may include non-transitory media in which data may be stored, and does not include carrier waves and / or temporary electronic signals that are propagated wirelessly or through a wired connection. Examples of non-transitory media may include, but are not limited to, disks or tapes, optical storage media such as compact discs (CDs) or digital versatile disks (DVDs), flash memory, memory, or storage devices. Computer-readable media may store thereon codes and / or machine-executable instructions, which may represent any combination of processes, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. A code segment may be coupled to another code segment or hardware circuit by passing and / or receiving information, data, independent variables, parameters, or memory contents. Information, independent variables, parameters, data, etc. may be passed, forwarded, or sent via any suitable means (including memory sharing, message passing, token passing, network transmission, etc.).

[0292] In some embodiments, computer-readable storage devices, media, and memories may include cables or wireless signals containing bit streams, etc. However, when referred to, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0293] Specific details are provided in the above description to provide a thorough understanding of the embodiments and examples provided herein. However, it will be appreciated by those skilled in the art that embodiments can be practiced without these specific details. For the sake of explanation, in some cases, the present technology can be presented as including separate functional blocks, which include functional blocks comprising devices, device components, steps or routines in the method embodied in software, or a combination of hardware and software. In addition to those components shown in the figure and / or described herein, additional components can also be used. For example, circuits, systems, networks, processes and other components can be shown as components in block diagram form so as not to obscure the embodiments in unnecessary details. In other cases, known circuits, processes, algorithms, structures and techniques can be shown without unnecessary details to avoid obscuring the embodiments.

[0294] Various embodiments may be described above as processes or methods, which are depicted as flow charts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. Although flow charts may describe operations as sequential processes, many operations may be performed in parallel or concurrently. In addition, the order of operations may be rearranged. A process terminates when its operations are completed, but may have additional steps not included in the figure. A process may correspond to a method, function, program, subroutine, subprogram, etc. When a process corresponds to a function, its termination may correspond to the function returning to a calling function or a main function.

[0295] Computer executable instructions stored or otherwise obtained from a computer readable medium can be used to implement the processes and methods according to the above examples. These instructions may include, for example, instructions and data that enable or otherwise configure a general-purpose computer, a special-purpose computer, or a processing device to perform a specific function or group of functions. Some of the computer resources used can be accessed through a network. Computer executable instructions can be, for example, binary files, intermediate format instructions (such as assembly language, firmware, source code, etc.). Examples of computer readable media that can be used to store instructions, information used, and / or information created during the method according to the examples include disks or optical disks, flash memory, USB devices provided with non-volatile memory, network storage devices, etc.

[0296] The equipment for implementing the process and method according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description language or any combination thereof, and may take any of various form factors. When implemented in software, firmware, middleware or microcode, the program code or code segment (e.g., computer program product) that performs the necessary tasks may be stored in a computer-readable or machine-readable medium. (Multiple) processors may perform the necessary tasks. Typical form factor examples include laptop computers, smart phones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mounted devices, stand-alone devices, etc. The functions described herein may also be embodied in peripheral devices or plug-in cards. As another example, this function may also be implemented on a circuit board between different chips, or implemented on different processes performed in a single device.

[0297] Instructions, media for transmitting such instructions, computing resources for executing the instructions, and other structures for supporting such computing resources are example components for providing the functionality described in this disclosure.

[0298] In the foregoing description, various aspects of the present application are described with reference to the specific embodiments of the present application, but those skilled in the art will recognize that the present application is not limited thereto. Therefore, although the illustrative embodiments of the present application have been described in detail herein, it should be understood that the inventive concept may be embodied and applied differently in other ways, and the appended claims are intended to be interpreted as including such variations, unless limited by the prior art. The various features and aspects of the above-mentioned application may be used individually or in combination. In addition, without departing from the broader spirit and scope of this specification, embodiments may be used in any number of environments and applications outside of those described herein. Therefore, the description and the accompanying drawings are considered to be illustrative rather than restrictive. For the purpose of illustration, the methods are described in a particular order. It should be understood that in alternative embodiments, the methods may be performed in an order different from the described order.

[0299] One of ordinary skill will understand that the less than ("<") and greater than (">") symbols or terms used herein may be replaced with less than or equal to ("≤") and greater than or equal to ("≥") symbols, respectively, without departing from the scope of this specification.

[0300] Where a component is described as being “configured to” perform certain operations, such configuration may be accomplished, for example, by designing electronic circuits or other hardware to perform the operations, by programming programmable electronic circuits (e.g., a microprocessor or other suitable electronic circuits) to perform the operations, or any combination thereof.

[0301] The phrase "coupled to" refers to any component that is physically connected directly or indirectly to another component, and / or any component that communicates directly or indirectly with another component (e.g., via a wired or wireless connection to other components, and / or other suitable communication interfaces).

[0302] Claim language or other language describing "at least one of" a set and / or "one or more" in a set indicates that one member of the set or multiple members of the set (in any combination) satisfies the claim. For example, claim language describing "at least one of A and B" refers to A, B, or A and B. In another example, claim language describing "at least one of A, B, and C" refers to A, B, C, or A and B, or A and C, or B and C, or A and B and C. The language "at least one of" a set and / or "one or more" in a set does not limit the set to the items listed in the set. For example, claim language describing "at least one of A and B" may mean A, B, or A and B, and may additionally include items not listed in the set of A and B.

[0303] The various illustrative logic blocks, modules, circuits and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, firmware or a combination thereof. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits and steps have been generally described above with respect to their functionality. Whether these functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. The technician can implement the described functions in different ways for each specific application, but this implementation decision should not be interpreted as causing deviations from the scope of the application.

[0304] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. These techniques may be implemented in any of a variety of devices (e.g., general-purpose computers, wireless communication device handsets, or integrated circuit devices with multiple uses, including applications in wireless communication device handsets and other devices). Any features described as modules or components may be implemented together in an integrated logic device, or individually as discrete but interoperable logic devices. If implemented in software, these techniques may be implemented at least in part by a computer-readable data storage medium, which includes a program code, which includes instructions for executing one or more of the above methods when executed. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may include a memory or data storage medium, such as a random access memory (RAM) (e.g., synchronous dynamic random access memory (SDRAM)), a read-only memory (ROM), a non-volatile random access memory (NVRAM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic or optical data storage medium, etc. Additionally or alternatively, these techniques may be implemented at least in part by a computer-readable communication medium that carries or transmits program code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer (e.g., a propagated signal or wave).

[0305] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor; however, in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors and a DSP core, or any other such configuration. Therefore, the term "processor" as used herein may refer to any of the aforementioned structures, any combination of the aforementioned structures, or any other structure or device suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within a dedicated software module or hardware module configured for encoding and decoding, or incorporated into a combined video encoder-decoder (codec).

[0306] Illustrative examples of the present disclosure include:

[0307] Example 1: A method for processing video data, the method comprising: obtaining a current block of video data; determining that at least one of a width and a height of the current block is greater than a size threshold; determining that a sub-partition split flag does not exist for the current block; and determining a value of the sub-partition split flag of the current block, the sub-partition split flag defining a split of the current block, wherein the value of the sub-partition split flag is determined based on whether the width or the height of the current block is greater than the size threshold.

[0308] Example 2: The method according to Example 1, wherein when the width of the current block is greater than the size threshold, the sub-partition split flag is determined to have a first value, the first value corresponding to the vertical split.

[0309] Example 3: The method according to any one of Examples 1 to 2, wherein when the height of the current block is greater than a size threshold, the sub-partition split flag is determined to have a second value corresponding to a horizontal split.

[0310] Example 4: A method according to Example 1, wherein when the width of the current block is greater than a size threshold and the height of the current block is less than the size threshold, the sub-partition split flag is determined to have a first value corresponding to a vertical split.

[0311] Example 5: A method according to any one of Examples 1 or 4, wherein when the height of the current block is greater than a size threshold and the width of the current block is less than the size threshold, the sub-partition split flag is determined to have a second value corresponding to a horizontal split.

[0312] Example 6: A method according to any one of Examples 1 to 5, wherein the sub-partition split flag is not signaled based on at least one of the width and height of the current block being greater than a size threshold.

[0313] Example 7: The method according to any one of Examples 1 to 6 further includes: dividing the current block vertically or horizontally into sub-partitions based on the sub-partition split flag.

[0314] Example 8: The method according to any one of Examples 1 to 7, wherein the current block is an intra-predicted block.

[0315] Example 9: A method according to any one of Examples 1 to 8, wherein the current block is a luma intra predicted block.

[0316] Example 10: An apparatus comprising a memory configured to store video data and a processor configured to process the video data according to the method of any of Examples 1 to 9.

[0317] Example 11: The apparatus of Example 10, wherein the apparatus comprises a decoder.

[0318] Example 12: An apparatus according to Example 10, wherein the apparatus includes an encoder.

[0319] Example 13: An apparatus according to any of Examples 10 to 12, wherein the apparatus is a mobile device.

[0320] Example 14: An apparatus according to any of Examples 10 to 13, wherein the apparatus comprises a display configured to display video data.

[0321] Example 15: An apparatus according to any of Examples 10 to 14, wherein the apparatus comprises a camera configured to capture one or more pictures.

[0322] Example 16: A computer readable medium having stored thereon instructions that, when executed by a processor, perform the method of any one of Examples 1 to 9.

[0323] Example 17: A method for processing video data, the method comprising: obtaining a current block of video data; determining that at least one of a width and a height of the current block is greater than a size threshold; and dividing the current block into sub-partitions, wherein the current block is divided until the width and height of the sub-partitions do not exceed the size threshold.

[0324] Example 18: The method of Example 17, wherein splitting the current block comprises reducing at least one of a width and a height that exceeds a size threshold.

[0325] Example 19: The method according to any one of Examples 17 to 18, wherein dividing the current block into sub-partitions comprises using a transform tree syntax structure when the sub-partition partition type is horizontal or vertical.

[0326] Example 20: The method according to any one of Examples 17 to 18 further includes modifying the value of a sub-partition number variable, which specifies the number of sub-partitions into which the block is divided, when at least one of the width and height of the encoded block is greater than a size threshold, wherein the modified value makes at least one of the width and height of the sub-partition not exceed the size threshold.

[0327] Example 21: A method according to any one of Examples 17 to 18, wherein the value of a subpartition mode flag associated with the current block constrains the subpartition to not exceed a size threshold of at least one of width and height, and the subpartition mode flag specifies whether the current block is divided into multiple block subpartitions.

[0328] Example 22: The method of Example 21, wherein the number of block subpartitions is specified by a number of subpartitions variable.

[0329] Example 23: An apparatus comprising a memory configured to store video data and a processor configured to process the video data according to the method of any of Examples 17 to 22.

[0330] Example 24: The apparatus of Example 23, wherein the apparatus comprises a decoder.

[0331] Example 25: An apparatus according to Example 23, wherein the apparatus comprises an encoder.

[0332] Example 26: An apparatus according to any of Examples 23 to 25, wherein the apparatus is a mobile device.

[0333] Example 27: An apparatus according to any of Examples 23 to 26, wherein the apparatus comprises a display configured to display video data.

[0334] Example 28: An apparatus according to any of Examples 23 to 27, wherein the apparatus comprises a camera configured to capture one or more pictures.

[0335] Example 29: A computer readable medium having stored thereon instructions that, when executed by a processor, perform the method described in any of Examples 17-22.

[0336] Example 30: A method of processing video data, the method comprising: obtaining a current block of video data; and encoding and decoding the current block based on a constraint, the constraint specifying that the current block does not cross a virtual pipeline data unit (VPDU) boundary.

[0337] Example 31: An apparatus according to Example 30, wherein the constraint specifies that when at least one of the width and height of the current block is greater than the maximum width or height of the VPDU, the value of the subpartition mode flag associated with the current block should be equal to a specific value, which specifies that the current block is not divided into block subpartitions.

[0338] Example 32: An apparatus comprising a memory configured to store video data and a processor configured to process the video data according to the apparatus of any of Examples 30-31.

[0339] Example 33: The apparatus of Example 32, wherein the apparatus comprises a decoder.

[0340] Example 34: An apparatus according to Example 32, wherein the apparatus comprises an encoder.

[0341] Example 35: An apparatus according to any of Examples 32 to 34, wherein the apparatus is a mobile device.

[0342] Example 36: An apparatus according to any of Examples 32 to 35, wherein the apparatus comprises a display configured to display video data.

[0343] Example 37: An apparatus according to any of Examples 32 to 36, wherein the apparatus comprises a camera configured to capture one or more pictures.

[0344] Example 38: A computer readable medium having stored thereon instructions that, when executed by a processor, perform the method described in any of Examples 30-31.

[0345] Article 1: An apparatus for encoding video data, the apparatus comprising: a memory; and a processor, the processor implemented in a circuit, configured to: obtain a current block of a picture of the video data; determine whether a width of the current block of the picture is greater than a width size threshold and whether a height of the current block is greater than at least one of a height size threshold, the width size threshold corresponding to a width of a data unit and the height size threshold corresponding to a height of the data unit; determine whether to signal an intra-sub-partition mode flag for the current block based on whether the width of the current block is greater than the width size threshold and whether the height of the current block is greater than at least one of the height size threshold, wherein a value of the intra-sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; and generate an encoded video bitstream including at least the current block based on determining whether the intra-sub-partition mode flag is to be signaled for the current block.

[0346] Clause 2: The apparatus of clause 1, wherein the width size threshold is equal to the height size threshold.

[0347] Clause 3: The apparatus of any of Clauses 1-2, wherein the width size threshold is different from the height size threshold.

[0348] Article 4: An apparatus according to any one of Articles 1-3, wherein the value of the intra-frame sub-partition mode flag is equal to the first value to specify that the current block is divided into rectangular transform block sub-partitions, and the value of the intra-frame sub-partition mode flag is equal to the second value to specify that the current block is not divided into rectangular transform block sub-partitions.

[0349] Article 5: An apparatus according to any one of Articles 1-4, wherein the processor is further configured to: determine that the width of the current block is greater than a width size threshold or the height of the current block is greater than a height size threshold; and based on the determination that the width of the current block is greater than the width size threshold or the height of the current block is greater than the height size threshold, determine that the value of the intra-frame sub-partition mode flag of the current block is equal to the second value.

[0350] Article 6: An apparatus according to any one of Articles 1-5, wherein the processor is further configured to: determine that the width of the current block is less than and equal to a width size threshold and the height of the current block is less than or equal to a height size threshold; and based on determining that the width of the current block is less than or equal to the width size threshold and the height of the current block is less than or equal to the height size threshold, determine to signal an intra-frame sub-partition mode flag for the current block (for example, equal to the first value or the second value).

[0351] Article 7: An apparatus according to any one of Articles 1-6, wherein the processor is further configured to: determine that the width of the current block is less than or equal to a size threshold and the height of the current block is less than or equal to the size threshold, the size threshold corresponding to the maximum transform block size; and based on the determination that the width of the current block is less than or equal to the size threshold and the height of the current block is less than or equal to the size threshold, determine to signal an intra-frame sub-partition mode flag for the current block (for example, equal to the first value or the second value).

[0352] Article 8: An apparatus according to any one of Articles 1-7, wherein the processor is further configured to: determine that the width of the current block is less than or equal to a size threshold or the height of the current block is less than or equal to the size threshold, the size threshold corresponding to the maximum transform block size; and based on the determination that the width of the current block is less than or equal to the size threshold or the height of the current block is less than or equal to the size threshold, determine that the value of the intra-frame sub-partition mode flag of the current block is a second value.

[0353] Article 9: An apparatus according to any one of Articles 1-8, wherein the processor is further configured to: determine that an intra-frame sub-partition mode flag is to be signaled for the current block; and based on the determination that the intra-frame sub-partition mode flag is to be signaled for the current block, include the intra-frame sub-partition mode flag of the current block in the encoded video bitstream.

[0354] Article 10: An apparatus according to any one of Articles 1-9, wherein the processor is further configured to: determine that at least one of the width and height of the current block is greater than a size threshold corresponding to the maximum transform block size; and based on the determination that the width or height of the current block is greater than the size threshold corresponding to the maximum transform block size, determine the value of an intra-frame sub-partition split flag of the current block, the intra-frame sub-partition split flag specifying whether the split type used to divide the current block is horizontal split or vertical split.

[0355] Article 11: An apparatus according to any one of Articles 1-10, wherein the processor is further configured to: determine that the width of the current block is greater than the size threshold; and based on the determination that the width of the current block is greater than the size threshold, determine a first segmentation value of the intra-frame sub-partition segmentation flag, the first segmentation value corresponding to the vertical segmentation type.

[0356] Article 12: An apparatus according to any one of Articles 1-11, wherein the processor is further configured to: determine that the height of the current block is greater than a size threshold; and based on the determination that the height of the current block is greater than the size threshold, determine a second segmentation value of the intra-frame sub-partition segmentation flag, the second segmentation value corresponding to the horizontal segmentation type.

[0357] Clause 13: The apparatus of any one of Clauses 1-12, wherein the processor is further configured to: vertically or horizontally divide the current block into sub-partitions based on a value of an intra-sub-partition split flag of the current block.

[0358] Clause 14: The apparatus of any of Clauses 1-13, wherein the current block is an intra-predicted block.

[0359] Clause 15: The apparatus of any of Clauses 1-14, wherein the data unit is a virtual pipeline data unit (VPDU).

[0360] Clause 16: The apparatus of any of Clauses 1-15, wherein the data unit is a transform block.

[0361] Clause 17: The apparatus of any of Clauses 1-16, wherein the width size threshold and the height size threshold are equal to the maximum transform block size.

[0362] Article 18: A method for decoding video data, the method comprising: obtaining an encoded video bitstream including video data; determining that there is no intra-frame sub-partition split flag for a current block of video data in the encoded video bitstream, the intra-frame sub-partition split flag specifying whether the split type of the intra-frame sub-partition mode used for the current block is horizontal or vertical; determining that at least one of the width and height of the current block is greater than a size threshold corresponding to a maximum transform block size; and determining a value of the intra-frame sub-partition split flag of the current block based on whether the width or height of the current block is greater than the size threshold.

[0363] Article 19: The method according to Article 18 also includes: determining that an intra-frame sub-partition mode flag is enabled for the current block; and based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, determining that there is no intra-frame sub-partition splitting flag in the encoded video bitstream, wherein based on the enabling of the intra-frame sub-partition mode flag for the current block, the current block is divided into rectangular transform block sub-partitions using the intra-frame sub-partition mode.

[0364] Article 20: The method according to any one of Articles 18-19 also includes: determining that the width of the current block is greater than the size threshold; and based on the determination that the width of the current block is greater than the size threshold, determining a first segmentation value of the intra-frame sub-partition segmentation flag, the first segmentation value corresponding to the vertical segmentation type.

[0365] Article 21: The method according to any one of Articles 18-20 also includes: using a vertical partitioning type to partition the current block into two or more sub-blocks, wherein based on the vertical partitioning type, the corresponding width of each of the two or more sub-blocks is smaller than the width of the current block.

[0366] Article 22: The method according to any one of Articles 18-21 also includes: determining that the height of the current block is greater than the size threshold; and based on the determination that the height of the current block is greater than the size threshold, determining a second segmentation value of the intra-frame sub-partition segmentation flag, the second segmentation value corresponding to the horizontal segmentation type.

[0367] Article 23: The method according to any one of Articles 18-22 also includes: using a horizontal partitioning type to partition the current block into two or more sub-blocks, wherein based on the horizontal partitioning type, the corresponding height of each of the two or more sub-blocks is less than the height of the current block.

[0368] Article 24: The method according to any one of Articles 18-23 also includes: determining that there is no intra-frame sub-partition mode flag in the encoded video bitstream for enabling the intra-frame sub-partition mode for the second block of video data; and determining the mode value of the intra-frame sub-partition mode flag for the second block based on the absence of the intra-frame sub-partition mode flag for the second block in the encoded video bitstream.

[0369] Article 25: A method according to any one of Articles 18-24, wherein the absence of an intra-frame sub-partition mode flag for the second block in the encoded video bitstream indicates that the width of the second block is greater than a width size threshold or the height of the second block is greater than at least one of a height size threshold, the width size threshold corresponds to the width of the data unit and the height size threshold corresponds to the height of the data unit.

[0370] Clause 26: A method as described in any of Clauses 18-25, wherein the width size threshold is equal to the height size threshold.

[0371] Clause 27: A method as described in any of Clauses 18-26, wherein the width size threshold is not equal to the height size threshold.

[0372] Clause 28: The method of any of Clauses 18-27, wherein the data unit is a virtual pipeline data unit (VPDU).

[0373] Clause 29: A method as recited in any of Clauses 18-28, wherein the data unit is a transform block.

[0374] Clause 30: A method as described in any of Clauses 18-29, wherein the width size threshold and the height size threshold are equal to the maximum transform block size.

[0375] Clause 31: A method as described in any of Clauses 18-30, wherein the current block is an intra-predicted block.

[0376] Article 32: The method according to any one of Articles 18-31 also includes: performing block-based partitioning of a block of video data to generate one or more codec blocks including a current block; determining to enable an intra-frame sub-partitioning mode for the current block; and based on determining to enable the intra-frame sub-partitioning mode for the current block, using the intra-frame sub-partitioning mode to divide the current block into two or more sub-blocks.

[0377] Clause 33: The method of any one of Clauses 18-32, further comprising: reconstructing the two or more sub-blocks by applying the corresponding two or more residual values ​​to the two or more sub-blocks.

[0378] Article 34: A device for decoding video data, the device comprising: a memory; and a memory implemented in a circuit, configured to: obtain an encoded video bitstream including video data; determine that there is no intra-frame sub-partition split flag for a current block of video data in the encoded video bitstream, the intra-frame sub-partition split flag specifying whether the split type of the intra-frame sub-partition mode for the current block is horizontal or vertical; determine that at least one of the width and height of the current block is greater than a size threshold corresponding to the maximum transform block size; and determine the value of the intra-frame sub-partition split flag of the current block based on whether the width or height of the current block is greater than the size threshold.

[0379] Article 35: An apparatus according to Article 34, wherein the processor is further configured to: determine that an intra-frame sub-partition mode flag is enabled for the current block; and based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, determine that there is no intra-frame sub-partition split flag in the encoded video bitstream, wherein based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, the current block is divided into rectangular transform block sub-partitions using the intra-frame sub-partition mode.

[0380] Article 36: An apparatus according to any one of Articles 34-35, wherein the processor is further configured to: determine that the width of the current block is greater than the size threshold; and based on the determination that the width of the current block is greater than the size threshold, determine a first segmentation value of the intra-frame sub-partition segmentation flag, the first segmentation value corresponding to the vertical segmentation type.

[0381] Article 37: An apparatus according to any one of Articles 34-36, wherein the processor is further configured to: use a vertical partitioning type to partition the current block into two or more sub-blocks, wherein based on the vertical partitioning type, the corresponding width of each of the two or more sub-blocks is smaller than the width of the current block.

[0382] Article 38: An apparatus according to any one of Articles 34-37, wherein the processor is further configured to: determine that the height of the current block is greater than a size threshold; and based on determining that the height of the current block is greater than the size threshold, determine a second segmentation value of the intra-frame sub-partition segmentation flag, the second segmentation value corresponding to the horizontal segmentation type.

[0383] Article 39: An apparatus according to any one of Articles 34-38, wherein the processor is further configured to: use a horizontal partitioning type to partition the current block into two or more sub-blocks, wherein based on the horizontal partitioning type, the corresponding height of each of the two or more sub-blocks is less than the height of the current block.

[0384] Article 40: An apparatus according to any one of Articles 34-39, wherein the processor is further configured to: determine that there is no intra-frame sub-partition mode flag in the encoded video bitstream for enabling the intra-frame sub-partition mode for the second block of video data; and determine the mode value of the intra-frame sub-partition mode flag for the second block based on the absence of the intra-frame sub-partition mode flag for the second block in the encoded video bitstream.

[0385] Article 41: An apparatus according to any one of Articles 34-40, wherein the absence of an intra-frame sub-partition mode flag for the second block in the encoded video bitstream indicates that the width of the second block is greater than a width size threshold or the height of the second block is greater than at least one of a height size threshold, the width size threshold corresponds to the width of a virtual pipeline data unit (VPDU) and the height size threshold corresponds to the height of the VPDU.

[0386] Clause 42: The apparatus of any of Clauses 34-41, wherein the width size threshold is equal to the height size threshold.

[0387] Clause 43: The apparatus of any of Clauses 34-42, wherein the width size threshold is not equal to the height size threshold.

[0388] Clause 44: The apparatus of any of Clauses 34-43, wherein the data unit is a virtual pipeline data unit (VPDU).

[0389] Clause 45: The apparatus of any of Clauses 34-44, wherein the data unit is a transform block.

[0390] Clause 46: The apparatus of any of Clauses 34-45, wherein the width size threshold and the height size threshold are equal to the maximum transform block size.

[0391] Clause 47: The apparatus of any of Clauses 34-43, wherein the current block is an intra-predicted block.

[0392] Article 48: An apparatus according to any one of Articles 34-44, wherein the processor is further configured to: perform block-based partitioning of a block of video data to generate one or more codec blocks including a current block; determine to enable an intra-frame sub-partitioning mode for the current block; and based on determining to enable the intra-frame sub-partitioning mode for the current block, divide the current block into two or more sub-blocks using the intra-frame sub-partitioning mode.

[0393] Clause 49: The apparatus of any one of Clauses 34-48, wherein the processor is further configured to reconstruct the two or more sub-blocks by applying the corresponding two or more residual values ​​to the two or more sub-blocks.

[0394] Clause 50: The apparatus of any of Clauses 34-49, wherein the apparatus comprises a mobile device having a camera for capturing one or more pictures.

[0395] Clause 51: The apparatus of any of Clauses 34-50, further comprising a display for displaying one or more images.

[0396] Article 52: A non-transitory computer-readable medium having instructions stored thereon, which, when executed by one or more processors, causes the one or more processors to: obtain an encoded video bitstream comprising video data; determine that there is no intra-frame sub-partition split flag for a current block of video data in the encoded video bitstream, the intra-frame sub-partition split flag specifying whether the split type of the intra-frame sub-partition mode used for the current block is horizontal or vertical; determine that at least one of the width and height of the current block is greater than a size threshold corresponding to a maximum transform block size; and determine a value of the intra-frame sub-partition split flag of the current block based on whether the width or height of the current block is greater than the size threshold.

[0397] Article 53: A non-transitory computer-readable medium according to Article 52, wherein the instructions further cause the processor to: determine that an intra-frame sub-partition mode flag is enabled for the current block; and based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, determine that there is no intra-frame sub-partition split flag in the encoded video bitstream, wherein based on the enabling of the intra-frame sub-partition mode flag for the current block, the current block is divided into rectangular transform block sub-partitions using the intra-frame sub-partition mode.

[0398] Article 54: A non-transitory computer-readable medium according to any one of Articles 52-53, wherein the instructions further cause the processor to: determine that the width of the current block is greater than a size threshold; and based on the determination that the width of the current block is greater than the size threshold, determine a first split value of the intra-frame sub-partition split flag, the first split value corresponding to the vertical split type.

[0399] Article 55: A non-transitory computer-readable medium according to any one of Articles 52-54, wherein the instructions further cause the processor to: use a vertical partitioning type to partition the current block into two or more sub-blocks, wherein based on the vertical partitioning type, the corresponding width of each of the two or more sub-blocks is less than the width of the current block.

[0400] Article 56: A non-transitory computer-readable medium according to any one of Articles 52-55, wherein the instructions further cause the processor to: determine that the height of the current block is greater than a size threshold; and based on determining that the height of the current block is greater than the size threshold, determine a second split value of the intra-frame sub-partition split flag, the second split value corresponding to the horizontal split type.

[0401] Article 57: A non-transitory computer-readable medium according to any one of Articles 52-56, wherein the instructions further cause the processor to: use a horizontal partitioning type to partition the current block into two or more sub-blocks, wherein based on the horizontal partitioning type, the corresponding height of each of the two or more sub-blocks is less than the height of the current block.

[0402] Article 58: A non-transitory computer-readable medium according to any one of Articles 52-57, wherein the instructions further cause the processor to: determine that there is no intra-frame sub-partition mode flag in the encoded video bitstream for enabling the intra-frame sub-partition mode for the second block of video data; and determine the mode value of the intra-frame sub-partition mode flag for the second block based on the absence of the intra-frame sub-partition mode flag for the second block in the encoded video bitstream.

[0403] Article 59: A non-transitory computer-readable medium according to any of Articles 52-58, wherein the absence of an intra-frame sub-partition mode flag for the second block in the encoded video bitstream indicates that the width of the second block is greater than a width size threshold or the height of the second block is greater than at least one of a height size threshold, the width size threshold corresponds to the width of a virtual pipeline data unit (VPDU) and the height size threshold corresponds to the height of the VPDU.

[0404] Clause 60: The non-transitory computer-readable medium of any of Clauses 52-59, wherein the width size threshold is equal to the height size threshold.

[0405] Clause 61: The non-transitory computer-readable medium of any of Clauses 52-60, wherein the width size threshold is not equal to the height size threshold.

[0406] Clause 62: The non-transitory computer-readable medium of any of Clauses 52-61, wherein the data unit is a virtual pipeline data unit (VPDU).

[0407] Clause 64: The non-transitory computer-readable medium of any of Clauses 52-62, wherein the data unit is a transform block.

[0408] Clause 65: The non-transitory computer-readable medium of any of Clauses 52-63, wherein the width size threshold and the height size threshold are equal to the maximum transform block size.

[0409] Clause 66: The non-transitory computer-readable medium of any of Clauses 52-64, wherein the current block is an intra-predicted block.

[0410] Article 67: A non-transitory computer-readable medium according to any of Articles 52-65, wherein the instructions further cause the processor to: perform block-based partitioning of a block of video data to generate one or more codec blocks including a current block; determine that an intra-frame sub-partitioning mode is enabled for the current block; and based on determining that the intra-frame sub-partitioning mode is enabled for the current block, use the intra-frame sub-partitioning mode to divide the current block into two or more sub-blocks.

[0411] Clause 68: The non-transitory computer-readable medium of any of Clauses 52-66, wherein the instructions further cause the processor to: reconstruct the two or more sub-blocks by applying the corresponding two or more residual values ​​to the two or more sub-blocks.

[0412] Article 69: An apparatus for decoding video data, the apparatus comprising: a component for obtaining an encoded video bitstream comprising video data; a component for determining that an intra-frame sub-partition split flag for a current block of video data does not exist in the encoded video bitstream, the intra-frame sub-partition split flag specifying whether the split type of the intra-frame sub-partition mode for the current block is horizontal or vertical; a component for determining that at least one of the width and height of the current block is greater than a size threshold corresponding to a maximum transform block size; and a component for determining a value of the intra-frame sub-partition split flag of the current block based on whether the width or height of the current block is greater than the size threshold.

[0413] Article 70: The device according to Article 69 also includes: a component for determining that an intra-frame sub-partition mode flag is enabled for the current block; and a component for determining that there is no intra-frame sub-partition splitting flag in the encoded video bitstream based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, wherein based on the determination that the intra-frame sub-partition mode flag is enabled for the current block, the current block is divided into rectangular transform block sub-partitions using the intra-frame sub-partition mode.

[0414] Article 71: The device according to any one of Articles 69-70 also includes: a component for determining that the width of the current block is greater than a size threshold; and a component for determining a first split value of the intra-frame sub-partition split flag based on the determination that the width of the current block is greater than the size threshold, wherein the first split value corresponds to a vertical split type.

[0415] Article 72: The device according to any one of Articles 69-71 also includes: a component for dividing the current block into two or more sub-blocks using a vertical partitioning type, wherein based on the vertical partitioning type, the corresponding width of each of the two or more sub-blocks is smaller than the width of the current block.

[0416] Article 73: The device according to any one of Articles 69-72 also includes: a component for determining that the height of the current block is greater than a size threshold; and a component for determining a second segmentation value of the intra-frame sub-partition segmentation flag based on the determination that the height of the current block is greater than the size threshold, wherein the second segmentation value corresponds to a horizontal segmentation type.

[0417] Article 74: The device according to any one of Articles 69-73 also includes: a component for splitting the current block into two or more sub-blocks using a horizontal split type, wherein based on the horizontal split type, the corresponding height of each of the two or more sub-blocks is less than the height of the current block.

[0418] Article 75: The device according to any one of Articles 69-74 also includes: a component for determining that there is no intra-frame sub-partition mode flag in the encoded video bitstream for enabling the intra-frame sub-partition mode for the second block of video data; and a component for determining the mode value of the intra-frame sub-partition mode flag of the second block based on the absence of the intra-frame sub-partition mode flag for the second block in the encoded video bitstream.

[0419] Article 76: An apparatus according to any one of Articles 69-75, wherein the absence of an intra-frame sub-partition mode flag for the second block in the encoded video bitstream indicates that the width of the second block is greater than a width size threshold or the height of the second block is greater than at least one of a height size threshold, the width size threshold corresponds to the width of a virtual pipeline data unit (VPDU) and the height size threshold corresponds to the height of the VPDU.

[0420] Clause 77: An apparatus as described in any of Clauses 69-76, wherein the width size threshold is equal to the height size threshold.

[0421] Clause 78: An apparatus as described in any of Clauses 69-77, wherein the width size threshold is not equal to the height size threshold.

[0422] Clause 79: The apparatus of any of Clauses 69-78, wherein the data unit is a virtual pipeline data unit (VPDU).

[0423] Clause 80: The apparatus of any of Clauses 69-79, wherein the data unit is a transform block.

[0424] Clause 81: An apparatus as described in any of Clauses 69-80, wherein the width size threshold and the height size threshold are equal to the maximum transform block size.

[0425] Clause 82: The apparatus of any of Clauses 69-81, wherein the current block is an intra-predicted block.

[0426] Article 83: The apparatus according to any one of Articles 69-82 further includes: a component for performing block-based partitioning of a block of video data to generate one or more codec blocks including a current block; a component for determining to enable an intra-frame sub-partitioning mode for the current block; and a component for partitioning the current block into two or more sub-blocks using the intra-frame sub-partitioning mode based on determining to enable the intra-frame sub-partitioning mode for the current block.

[0427] Clause 84: The apparatus of any one of Clauses 69-83, further comprising: means for reconstructing two or more sub-blocks by applying corresponding two or more residual values ​​to the two or more sub-blocks.

Claims

1. A device for encoding video data, the device comprising: Memory; as well as The processor implemented in the circuit is configured to: Get the current block of the picture of the video data; Determining that a width of the current block of the picture is greater than a maximum transform block size, and determining that a height of the current block is less than or equal to the maximum transform block size; Based on determining that the width of the current block is greater than the maximum transform block size and determining that the height of the current block is less than or equal to the maximum transform block size, determining not to signal a first intra sub-partition mode flag for the current block, wherein the first intra sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; as well as Based on determining not to signal the first intra sub-partition mode flag for the current block, not including the first intra sub-partition mode flag in a coded video bitstream including the current block.

2. The device according to claim 1, wherein: The value of the first intra subpartition mode flag equal to a first value specifies that the current block is divided into rectangular transform block subpartitions, and the value of the first intra subpartition mode flag equal to a second value specifies that the current block is not divided into rectangular transform block subpartitions.

3. The device according to claim 2, wherein: The processor is further configured to: Based on determining not to signal the first intra sub-partition mode flag for the current block, determining a value of the first intra sub-partition mode flag for the current block to be the second value.

4. The device according to claim 1, wherein: The processor is further configured to: Get the second block of video data; determining that a width of the second block is less than or equal to the maximum transform block size, and determining that a height of the second block is greater than the maximum transform block size; Based on determining that the width of the second block is less than or equal to the maximum transform block size and determining that the height of the second block is greater than the maximum transform block size, determining not to signal a second intra sub-partition mode flag for the second block, wherein the second intra sub-partition mode flag indicates whether the second block is divided into rectangular transform block sub-partitions; as well as Based on determining not to signal the second intra sub-partition mode flag for the second block, not including the second intra sub-partition mode flag in a coded video bitstream including the current block and the second block.

5. The device according to claim 4, wherein: The processor is further configured to: Get the third block of video data; determining that a width of the third block is less than or equal to the maximum transform block size and determining that a height of the third block is less than or equal to the maximum transform block size; Based on determining that the width of the third block is less than or equal to the maximum transform block size and determining that the height of the third block is less than or equal to the maximum transform block size, determining to signal a third intra sub-partition mode flag for the third block, wherein the third intra sub-partition mode flag indicates whether the third block is divided into rectangular transform block sub-partitions; as well as Based on determining to signal the third intra sub-partition mode flag for the third block, the third intra sub-partition mode flag is included in a coded video bitstream including the current block, the second block, and the third block.

6. The device according to claim 5, wherein: The processor is further configured to: Based on determining that the third intra sub-partition mode flag is to be signaled for the third block, a value of an intra sub-partition split flag of the third block is determined, the intra sub-partition split flag specifying whether a partition type for dividing the third block is horizontal or vertical.

7. The device according to claim 6, wherein: The processor is further configured to: The third block is divided into sub-partitions vertically or horizontally based on a value of the intra-sub-partition split flag of the third block.

8. The device according to claim 1, wherein: The current block is an intra-predicted block.

9. The device according to claim 1, wherein: The transform block corresponding to the maximum transform block size is a virtual pipeline data unit (VPDU).

10. An apparatus for decoding video data, the apparatus comprising: Memory; as well as The processor implemented in the circuit is configured to: Get the current block of the picture of the video data; Determining that a width of the current block of the picture is greater than a value indicating a maximum transform block size, and determining that a height of the current block is less than or equal to the value indicating the maximum transform block size; determining that a first intra sub-partition mode flag is not received for the current block based on determining that a width of the current block is greater than the value indicating the maximum transform block size and determining that a height of the current block is less than or equal to the value indicating the maximum transform block size, wherein the first intra sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; as well as Based on determining that the first intra sub-partition mode flag is not received for the current block, processing the current block.

11. The device according to claim 10, wherein: The first intra sub-partition mode flag being equal to a first value specifies that the current block is partitioned into rectangular transform block sub-partitions, and the first intra sub-partition mode flag being equal to a second value specifies that the current block is not partitioned into rectangular transform block sub-partitions.

12. The apparatus according to claim 11, wherein the processor is further configured to: Based on determining that the first intra sub-partition mode flag is not received for the current block, determining that the value of the first intra sub-partition mode flag of the current block is equal to the second value.

13. The device according to claim 10, wherein: The processor is further configured to: Obtaining a second block of video data, the second block being different from the current block; determining that a width of the second block is less than or equal to the value indicating the maximum transform block size, and determining that a height of the second block is greater than the value indicating the maximum transform block size; determining that a second intra sub-partition mode flag is not received for the second block based on determining that a width of the second block is less than or equal to the value indicating the maximum transform block size and determining that a height of the second block is greater than the value indicating the maximum transform block size, wherein the second intra sub-partition mode flag indicates whether the second block is divided into rectangular transform block sub-partitions, and wherein the second intra sub-partition mode flag is different from the first intra sub-partition mode flag; as well as Based on determining that the second intra sub-partition mode flag is not received for the second block, the second block is processed.

14. The device according to claim 13, wherein: The processor is further configured to: Obtaining a third block of video data, the third block being different from the current block and different from the second block; determining that a width of the third block is less than or equal to the value indicating the maximum transform block size, and determining that a height of the third block is less than or equal to the value indicating the maximum transform block size; determining that a third intra sub-partition mode flag is received for the third block based on determining that a width of the third block is less than or equal to the value indicating the maximum transform block size and determining that a height of the third block is less than or equal to the value indicating the maximum transform block size, wherein the third intra sub-partition mode flag indicates whether the third block is divided into rectangular transform block sub-partitions, and wherein the third intra sub-partition mode flag is different from the first intra sub-partition mode flag and different from the second intra sub-partition mode flag; as well as Based on determining that the third intra sub-partition mode flag is received for the third block, a value of an intra sub-partition split flag of the third block is determined, the intra sub-partition split flag specifying whether a partition type for dividing the third block is horizontal or vertical.

15. The device according to claim 14, wherein: The processor is further configured to: The third block is divided into sub-partitions vertically or horizontally based on the value of the intra-sub-partition split flag of the third block.

16. The device according to claim 10, wherein: The current block is an intra-predicted block.

17. The device according to claim 10, wherein: The transform block corresponding to the maximum transform block size is a virtual pipeline data unit (VPDU).

18. A method for encoding video data, comprising: Get the current block of the picture of the video data; Determining that a width of the current block of the picture is greater than a maximum transform block size, and determining that a height of the current block is less than or equal to the maximum transform block size; Based on determining that the width of the current block is greater than the maximum transform block size and determining that the height of the current block is less than or equal to the maximum transform block size, determining not to signal a first intra sub-partition mode flag for the current block, wherein the first intra sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; as well as Based on determining not to signal the first intra sub-partition mode flag for the current block, not including the first intra sub-partition mode flag in a coded video bitstream including the current block.

19. The method according to claim 18, wherein: The value of the first intra subpartition mode flag equal to a first value specifies that the current block is divided into rectangular transform block subpartitions, and the value of the first intra subpartition mode flag equal to a second value specifies that the current block is not divided into rectangular transform block subpartitions.

20. The method according to claim 19, further comprising: Based on determining not to signal the first intra sub-partition mode flag for the current block, determining a value of the first intra sub-partition mode flag for the current block to be the second value.

21. The method of claim 18, further comprising: Get the second block of video data; determining that a width of the second block is less than or equal to the maximum transform block size, and determining that a height of the second block is greater than the maximum transform block size; Based on determining that the width of the second block is less than or equal to the maximum transform block size and determining that the height of the second block is greater than the maximum transform block size, determining not to signal a second intra sub-partition mode flag for the second block, wherein the second intra sub-partition mode flag indicates whether the second block is divided into rectangular transform block sub-partitions; as well as Based on determining not to signal the second intra sub-partition mode flag for the second block, not including the second intra sub-partition mode flag in a coded video bitstream including the current block and the second block.

22. The method according to claim 21, further comprising: Get the third block of video data; Determine that a width of the third block is less than or equal to the maximum transform block size, and determine that a height of the third block is less than or equal to the maximum transform block size; Based on determining that the width of the third block is less than or equal to the maximum transform block size and determining that the height of the third block is less than or equal to the maximum transform block size, determining to signal a third intra sub-partition mode flag for the third block, wherein the third intra sub-partition mode flag indicates whether the third block is divided into rectangular transform block sub-partitions; as well as Based on determining to signal the third intra sub-partition mode flag for the third block, the third intra sub-partition mode flag is included in a coded video bitstream including the current block, the second block, and the third block.

23. The method according to claim 22, further comprising: Based on determining that the third intra sub-partition mode flag is to be signaled for the third block, a value of an intra sub-partition split flag of the third block is determined, the intra sub-partition split flag specifying whether a partition type for dividing the third block is horizontal or vertical.

24. The method according to claim 23, further comprising: The third block is divided into sub-partitions vertically or horizontally based on a value of the intra-sub-partition split flag of the third block.

25. The method of claim 18, wherein: The current block is an intra-predicted block.

26. The method of claim 18, wherein: The transform block corresponding to the maximum transform block size is a virtual pipeline data unit (VPDU).

27. A method for decoding video data, comprising: Get the current block of the picture of the video data; Determining that a width of the current block of the picture is greater than a value indicating a maximum transform block size, and determining that a height of the current block is less than or equal to the value indicating the maximum transform block size; determining that a first intra sub-partition mode flag is not received for the current block based on determining that a width of the current block is greater than the value indicating the maximum transform block size and determining that a height of the current block is less than or equal to the value indicating the maximum transform block size, wherein the first intra sub-partition mode flag indicates whether the current block is divided into rectangular transform block sub-partitions; as well as Based on determining that the first intra sub-partition mode flag is not received for the current block, processing the current block.

28. The method according to claim 27, wherein: The first intra sub-partition mode flag being equal to a first value specifies that the current block is partitioned into rectangular transform block sub-partitions, and the first intra sub-partition mode flag being equal to a second value specifies that the current block is not partitioned into rectangular transform block sub-partitions.

29. The method according to claim 28, further comprising: Based on determining that the first intra sub-partition mode flag is not received for the current block, determining that the value of the first intra sub-partition mode flag of the current block is equal to the second value.

30. The method of claim 27, further comprising: Obtaining a second block of video data, the second block being different from the current block; determining that a width of the second block is less than or equal to the value indicating the maximum transform block size, and determining that a height of the second block is greater than the value indicating the maximum transform block size; determining that a second intra sub-partition mode flag is not received for the second block based on determining that a width of the second block is less than or equal to the value indicating the maximum transform block size and determining that a height of the second block is greater than the value indicating the maximum transform block size, wherein the second intra sub-partition mode flag indicates whether the second block is divided into rectangular transform block sub-partitions, and wherein the second intra sub-partition mode flag is different from the first intra sub-partition mode flag; as well as Based on determining that the second intra sub-partition mode flag is not received for the second block, the second block is processed.

31. The method of claim 30, further comprising: Obtaining a third block of video data, the third block being different from the current block and different from the second block; determining that a width of the third block is less than or equal to the value indicating the maximum transform block size, and determining that a height of the third block is less than or equal to the value indicating the maximum transform block size; determining that a third intra sub-partition mode flag is received for the third block based on determining that a width of the third block is less than or equal to the value indicating the maximum transform block size and determining that a height of the third block is less than or equal to the value indicating the maximum transform block size, wherein the third intra sub-partition mode flag indicates whether the third block is divided into rectangular transform block sub-partitions, and wherein the third intra sub-partition mode flag is different from the first intra sub-partition mode flag and different from the second intra sub-partition mode flag; as well as Based on determining that the third intra sub-partition mode flag is received for the third block, a value of an intra sub-partition split flag of the third block is determined, the intra sub-partition split flag specifying whether a partition type for dividing the third block is horizontal or vertical.

32. The method of claim 31 further comprising: The third block is divided into sub-partitions vertically or horizontally based on the value of the intra-sub-partition split flag of the third block.

33. The method of claim 27, wherein: The current block is an intra-predicted block.

34. The method of claim 27, wherein: The transform block corresponding to the maximum transform block size is a virtual pipeline data unit (VPDU).

35. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 18-34.

Images