Method and apparatus for dividing blocks at image boundaries

Abstract

The present disclosure provides a video processing method and apparatus. The example method includes: determining whether a coding block includes samples outside an image boundary; and in response to the coding block being determined to include samples outside an image boundary, regardless of the value of a first parameter, performing quadtree segmentation of the coding block, wherein the first parameter indicates whether the quadtree is allowed to be used to segment the coding block.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to U.S. Provisional Application No. 62 / 948,856 filed on December 17, 2019, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to video processing, and more particularly, to methods and apparatus for block partitioning at image boundaries. Background Art

[0004] A video is a set of static images (or "frames") that capture visual information. In order to reduce storage memory and transmission bandwidth, the video can be compressed before storage or transmission, and then decompressed before display. The compression process is usually called encoding, and the decompression process is usually called decoding. There are many video coding formats that use standardized video coding techniques, the most common of which are based on prediction, transform, quantization, entropy coding, and loop filtering. Video coding standards, such as the High Efficiency Video Coding (HEVC / H.265) standard, the Versatile Video Coding (VVC / H.266) standard, and the AVS standard, specify specific video coding formats and are developed by standardization organizations. As more and more advanced video coding technologies are adopted in video standards, the coding efficiency of new video coding standards is getting higher and higher. Summary of the invention

[0005] In some embodiments, an exemplary video processing method includes: determining whether a coding block includes samples outside a picture boundary; and in response to the coding block being determined to include samples outside a picture boundary, performing quadtree segmentation of the coding block regardless of a value of a first parameter, wherein the first parameter indicates whether the use of the quadtree to segment the coding block is allowed.

[0006] In some embodiments, an exemplary video processing apparatus includes at least one memory for storing instructions and at least one processor, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform: determining whether a coding block includes samples outside a picture boundary; and in response to the coding block being determined to include samples outside a picture boundary, performing quadtree segmentation of the coding block regardless of a value of a first parameter, wherein the first parameter indicates whether the quadtree is allowed to be used to segment the coding block.

[0007] In some embodiments, an exemplary non-transitory computer-readable storage medium stores an instruction set. The instruction set can be executed by one or more processing devices to cause a video processing device to perform: determining whether a coding block includes samples outside a picture boundary, and in response to the coding block being determined to include samples outside a picture boundary, performing quadtree segmentation of the coding block regardless of a value of a first parameter, wherein the first parameter indicates whether the quadtree is allowed to be used to segment the coding block. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Embodiments and various aspects of the present disclosure are illustrated in the following detailed description and accompanying drawings.The various features illustrated in the drawings are not drawn to scale.

[0009] Figure 1 is a schematic diagram of the structure of an exemplary video sequence according to some embodiments of the present disclosure.

[0010] Figure 2A is a schematic diagram illustrating an exemplary encoding process of a hybrid video coding system according to an embodiment of the present disclosure.

[0011] Figure 2B is a schematic diagram illustrating another exemplary encoding process of a hybrid video encoding system according to an embodiment of the present disclosure.

[0012] Figure 3A is a schematic diagram illustrating an exemplary decoding process of a hybrid video coding system according to an embodiment of the present disclosure.

[0013] Figure 3B is a schematic diagram illustrating another exemplary decoding process of a hybrid video coding system according to an embodiment of the present disclosure.

[0014] Figure 4 is a block diagram of an exemplary apparatus for encoding or decoding a video according to some embodiments of the present disclosure.

[0015] Figure 5 is a schematic diagram showing an example of a multi-type tree splitting mode according to some embodiments of the present disclosure.

[0016] Figure 6 is a diagram illustrating an exemplary signaling mechanism for partition splitting information in a quadtree (QT) having a nested multi-type tree coding tree structure according to some embodiments of the present disclosure.

[0017] Figure 7 An exemplary Table 1 is shown, which illustrates an exemplary multi-type tree partition mode (MttSplitMode) derivation based on a multi-type tree syntax element according to some embodiments of the present disclosure.

[0018] Figure 8is a diagram illustrating examples of disallowed ternary tree (TT) and binary tree (BT) partitioning according to some embodiments of the present disclosure.

[0019] Fig. 9 is a schematic diagram showing exemplary blocks divided on image boundaries according to some embodiments of the present disclosure.

[0020] Fig.10 An exemplary Table 2 is shown according to some embodiments of the present disclosure, which shows an exemplary specification of parallel ternary tree partitioning (parallelTtSplit) and coding block size (cbSize) based on a binary partitioning mode (btSplit).

[0021] Fig.11 Exemplary Table 3 is shown, which shows an exemplary specification of cbSize based on the ternary tree partitioning mode (ttSplit) according to some embodiments of the present disclosure.

[0022] Fig.12 An exemplary Table 4 is shown, which shows an exemplary coding tree syntax according to some embodiments of the present disclosure.

[0023] Fig.13 is a schematic diagram illustrating an example of a multi-type tree splitting mode indicated by MttSplitMode according to some embodiments of the present disclosure.

[0024] Fig.14 Exemplary Table 5 is shown, which shows an exemplary specification of MttSplitMode according to some embodiments of the present disclosure.

[0025] Fig.15 An exemplary Table 6 is shown, which illustrates an exemplary sequence parameter set RBSP syntax according to some embodiments of the present disclosure.

[0026] Fig.16 An exemplary Table 7 is shown, which shows an exemplary image header RBSP syntax according to some embodiments of the present disclosure.

[0027] Fig.17 is a diagram illustrating an exemplary block that does not allow QT, TT, or BT partitioning at an image boundary according to some embodiments of the present disclosure.

[0028] Fig.18 is a diagram illustrating an exemplary block that does not allow QT, TT, or BT partitioning at an image boundary according to some embodiments of the present disclosure.

[0029] Fig.19 is a schematic diagram illustrating an example of using BT and TT partitioning according to some embodiments of the present disclosure.

[0030] Fig. 20 A flowchart of an exemplary video processing method according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0031] Reference will now be made in detail to exemplary embodiments, examples of which are shown in the accompanying drawings. The following description refers to the accompanying drawings, in which the same numbers in different drawings represent the same or similar elements unless otherwise specified. The embodiments set forth in the following description of the exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with aspects related to the present disclosure as described in the appended claims. Specific aspects of the present disclosure are described in more detail below. If there is a conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall prevail.

[0032] The Joint Video Experts Group (JVET) of the ITU-T Video Coding Experts Group (ITU-T VCEG) and the ISO / IEC Moving Picture Experts Group (ISO / IEC MPEG) is currently developing the Versatile Video Coding (VVC / H.266) standard. The VVC standard aims to double the compression efficiency of its predecessor, the High Efficiency Video Coding (HEVC / H.265) standard. In other words, VVC aims to achieve the same subjective quality as HEVC / H.265 using half the bandwidth.

[0033] In order to achieve the same subjective quality as HEVC / H.265 using half the bandwidth, JVET has been developing technologies beyond HEVC using the Joint Exploration Model (JEM) reference software. As coding technologies are incorporated into JEM, JEM achieves higher coding performance than HEVC.

[0034] The VVC standard was developed recently and continues to include more coding techniques that provide better compression performance. VVC is based on the same hybrid video coding system that has been used in modern video compression standards such as HEVC, H.264 / AVC, MPEG2, H.263, etc.

[0035] A video is a set of static images (or "frames") arranged in time sequence to store visual information. These images can be captured and stored in time sequence using a video capture device (e.g., a camera), and such images in time sequence can be displayed using a video playback device (e.g., a television, a computer, a smartphone, a tablet computer, a video player, or any end-user terminal with display capabilities). In addition, in some applications, the video capture device can send the captured video to a video playback device (e.g., a computer with a monitor) in real time, such as for monitoring, conferencing, or live broadcasting.

[0036] In order to reduce the storage space and transmission bandwidth required for such applications, the video can be compressed before storage and transmission, and decompressed before display. Compression and decompression can be implemented by software executed by a processor (e.g., a processor of a general-purpose computer) or dedicated hardware. The module for compression is generally referred to as an "encoder", and the module for decompression is generally referred to as a "decoder". Encoders and decoders can be collectively referred to as "codecs". Encoders and decoders can be implemented as any of various suitable hardware, software, or combinations thereof. For example, the hardware implementation of encoders and decoders can include circuits such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, or any combination thereof. The software implementation of encoders and decoders can include program code, computer executable instructions, firmware, or any suitable computer-implemented algorithm or process fixed in a computer-readable medium. Video compression and decompression can be implemented by various algorithms or standards, such as MPEG-1, MPEG-2, MPEG-4, H.26x series, etc. In some applications, a codec can decompress a video from a first coding standard and recompress the decompressed video using a second coding standard, in which case the codec can be referred to as a "transcoder".

[0037] The video encoding process can identify and retain useful information that can be used to reconstruct the image, and ignore unimportant reconstruction information. If the ignored unimportant information cannot be completely reconstructed, such an encoding process can be called "lossy". Otherwise, it can be called "lossless". Most encoding processes are lossy, which is a trade-off to reduce the required storage space and transmission bandwidth.

[0038] Useful information about the image being encoded (referred to as the "current image") includes changes relative to a reference image (e.g., a previously encoded and reconstructed image). Such changes can include changes in pixel position, brightness, or color, with position changes being of greatest interest. Changes in the position of a group of pixels representing an object can reflect the object's motion between the reference image and the current image.

[0039] A picture that is encoded without reference to another picture (i.e. it is its own reference picture) is called an "I-picture". A picture that is encoded using a previous picture as a reference picture is called a "P-picture", and a picture that is encoded using both a previous picture and a future picture as reference pictures is called a "B-picture" (the reference is "bidirectional").

[0040] Figure 1The structure of an example video sequence 100 according to some embodiments of the present disclosure is shown. The video sequence 100 may be a live video or a video that has been captured and archived. The video 100 may be a real-life video, a computer-generated video (e.g., a computer game video), or a combination of the two (e.g., a real video with an augmented reality effect). The video sequence 100 may be input from a video capture device (e.g., a camera), a video archive containing previously captured videos (e.g., a video file stored in a storage device), or a video feed interface (e.g., a video broadcast transceiver) that receives video from a video content provider.

[0041] like Figure 1 As shown, video sequence 100 may include a series of images arranged in time along a timeline, including images 102, 104, 106, and 108. Images 102-106 are consecutive, and there are more images between images 106 and 108. Figure 1 , picture 102 is an I-picture, and its reference picture is picture 102 itself. Picture 104 is a P-picture, and its reference picture is picture 102, as indicated by the arrow. Picture 106 is a B-picture, and its reference pictures are pictures 104 and 108, as indicated by the arrow. In some embodiments, the reference picture of a picture (e.g., picture 104) may not be immediately before or after the picture. For example, the reference picture of picture 104 may be the picture before picture 102. It should be noted that the reference pictures of pictures 102-106 are merely examples, and the present disclosure is not limited to such pictures. Figure 1 Example of a reference image shown.

[0042] Typically, video codecs do not encode or decode the entire image at once due to the computational complexity of the encoding and decoding tasks. Instead, they may segment the image into basic segments and encode or decode the image segments one by one. In this disclosure, such basic segments are referred to as basic processing units ("BPUs"). For example, Figure 1Structure 110 in shows an example structure of an image of video sequence 100 (e.g., any of images 102-108). In structure 110, the image is divided into 4×4 basic processing units, whose boundaries are shown as dashed lines. In some embodiments, the basic processing unit may be referred to as a "macroblock" in some video coding standards (e.g., the MPEG family, H.261, H.263, or H.264 / AVC), or as a "coding tree unit" ("CTU") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing unit may have a variable size in the image, such as 128×128, 64×64, 32×32, 16×16, 4×8, 16×32, or pixels of any shape and size. The size and shape of the basic processing unit may be selected for the image based on a balance between coding efficiency and the level of detail to be maintained in the basic processing unit.

[0043] A basic processing unit may be a logical unit that may include a set of different types of video data stored in a computer memory (e.g., in a video frame buffer). For example, a basic processing unit of a color image may include a luma component (Y) representing achromatic luminance information, one or more chroma components (e.g., Cb and Cr) representing color information, and associated syntax elements, where the luma and chroma components may have the same size as the basic processing unit. In some video coding standards (e.g., H.265 / HEVC or H.266 / VVC), luma and chroma components may be referred to as "coding tree blocks" ("CTBs"). Any operation performed on a basic processing unit may be repeated for each of its luma and chroma components.

[0044] Video encoding has multiple stages of operation, examples of which are Figure 2A-2B and Figure 3A-3BAs shown. For each stage, the size of the basic processing unit may still be too large for processing, so it can be further divided into segments referred to as "basic processing subunits" in this disclosure. In some embodiments, the basic processing subunit may be referred to as a "block" in some video coding standards (e.g., MPEG family, H.261, H.263 or H.264 / AVC), or as a "coding unit" ("CU") in some other video coding standards (e.g., H.265 / HEVC or H.266 / VVC). The basic processing subunit may have the same size as the basic processing unit or a smaller size than the basic processing unit. Similar to the basic processing unit, the basic processing subunit is also a logical unit, which may include a set of different types of video data (e.g., Y, Cb, Cr and associated syntax elements) stored in a computer memory (e.g., in a video frame buffer). Any operation performed on the basic processing subunit can be repeated for each of its luminance and chrominance components. It should be noted that this division can be performed to a further level according to processing needs. It should also be noted that different schemes can be used to divide the basic processing unit at different stages.

[0045] For example, in the mode decision phase (an example of which is in Figure 2B As shown in , the encoder can decide what prediction mode (e.g., intra prediction or inter prediction) to use for a basic processing unit, which may be too large to make such a decision. The encoder can divide the basic processing unit into multiple basic processing sub-units (e.g., CUs in H.265 / HEVC or H.266 / VVC) and decide the prediction type for each individual basic processing sub-unit.

[0046] For another example, in the prediction phase (the example in Figure 2A-2B ), the encoder can perform prediction operations at the level of basic processing sub-units (e.g., CUs). However, in some cases, the basic processing sub-units may still be too large to process. The encoder can further divide the basic processing sub-units into smaller segments (e.g., called "prediction blocks" or "PBs" in H.265 / HEVC or H.266 / VVC), at which prediction operations can be performed.

[0047] For another example, in the transformation phase (the example in Figure 2A-2B), the encoder can perform transform operations on the residual basic processing sub-unit (e.g., CU). However, in some cases, the basic processing sub-unit may still be too large to process. The encoder can further divide the basic processing sub-unit into smaller segments (e.g., called "transform blocks" or "TBs" in H.265 / HEVC or H.266 / VVC), at which level transform operations can be performed. It should be noted that the division scheme of the same basic processing sub-unit can be different in the prediction stage and the transform stage. For example, in H.265 / HEVC or H.266 / VVC, the prediction blocks and transform blocks of the same CU can have different sizes and numbers.

[0048] exist Figure 1 In the structure 110, the basic processing unit 112 is further divided into 3×3 basic processing sub-units, whose boundaries are shown by dotted lines. Different basic processing units of the same image can be divided into basic processing sub-units in different schemes.

[0049] In some embodiments, in order to provide the ability of parallel processing of video encoding and decoding and fault tolerance, the image can be divided into regions for processing, so that for the region of the image, the encoding or decoding process can be independent of information from any other region of the image. In other words, each region of the image can be processed independently. By doing so, the codec can process different regions of the image in parallel, thereby improving coding efficiency. In addition, when the data of a region is damaged during processing or lost during network transmission, the codec can correctly encode or decode other regions of the same image without relying on the damaged or lost data, thereby providing fault tolerance. In some video coding standards, the image can be divided into different types of regions. For example, H.265 / HEVC and H.266 / VVC provide two types of regions: "slices" and "tiles". It should also be noted that different images of the video sequence 100 can have different partitioning schemes for dividing the image into regions.

[0050] For example, in Figure 1 In FIG. 1 , the structure 110 is divided into three regions 114, 116, and 118, whose boundaries are shown as solid lines inside the structure 110. The region 114 includes four basic processing units. The regions 116 and 118 each include six basic processing units. It should be noted that Figure 1 The basic processing units, basic processing sub-units, and regions of the structure 110 are merely examples, and the present disclosure does not limit the embodiments thereof.

[0051] Figure 2A FIG. 2 is a schematic diagram of an exemplary encoding process 200A according to an embodiment of the present disclosure. For example, the encoding process 200A may be performed by an encoder. Figure 2AAs shown, the encoder may encode the video sequence 202 into a video bitstream 228 according to process 200A. Figure 1 The video sequence 100 in FIG. 200 may include a set of images (referred to as “original images”) arranged in time sequence. Figure 1 In the structure 110 in the video sequence 202, each original image of the video sequence 202 can be divided into basic processing units, basic processing sub-units or regions for processing by the encoder. In some embodiments, the encoder can perform process 200A at the level of the basic processing unit for each original image of the video sequence 202. For example, the encoder can perform process 200A in an iterative manner, wherein the encoder can encode the basic processing unit in one iteration of process 200A. In some embodiments, the encoder can perform process 200A in parallel for regions (e.g., regions 114-118) of each original image of the video sequence 202.

[0052] refer to Figure 2A , the encoder may feed the basic processing unit of the original image of the video sequence 202 (referred to as the "original BPU") to the prediction stage 204 to generate prediction data 206 and prediction BPU 208. The encoder may subtract the predicted BPU 208 from the original BPU to generate a residual BPU 210. The encoder may feed the residual BPU 210 to the transform stage 212 and the quantization stage 214 to generate quantized transform coefficients 216. The encoder may feed the prediction data 206 and the quantized transform coefficients 216 to the binary encoding stage 226 to generate a video bitstream 228. The components 202, 204, 206, 208, 210, 212, 214, 216, 226, and 228 may be referred to as the "forward path". During the process 200A, after the quantization stage 214, the encoder may feed the quantized transform coefficients 216 to the inverse quantization stage 218 and the inverse transform stage 220 to generate a reconstructed residual BPU 222. The encoder may add the reconstructed residual BPU 222 to the predicted BPU 208 to generate a prediction reference 224, which is used in the prediction phase 204 of the next iteration of the process 200A. The components 218, 220, 222, and 224 of the process 200A may be referred to as a "reconstruction path." The reconstruction path may be used to ensure that both the encoder and the decoder use the same reference data for prediction.

[0053] The encoder can iteratively perform process 200A to encode each original BPU (in the forward path) of the encoded original image and generate a prediction reference 224 for encoding the next original BPU (in the reconstruction path) of the encoded original image. After encoding all the original BPUs of the original image, the encoder can proceed to encode the next image in the video sequence 202.

[0054] Referring to process 200A, an encoder may receive a video sequence 202 generated by a video acquisition device (eg, a camera). As used herein, the term "receiving" may refer to any action of receiving, inputting, acquiring, retrieving, obtaining, reading, accessing, or inputting data in any manner.

[0055] In the prediction phase 204, at the current iteration, the encoder may receive the original BPU and the prediction reference 224 and perform a prediction operation to generate the prediction data 206 and the prediction BPU 208. The prediction reference 224 may be generated from the reconstruction path of the previous iteration of the process 200A. The purpose of the prediction phase 204 is to reduce information redundancy by extracting the prediction data 206 that can be used to reconstruct the original BPU into the prediction BPU 208 from the prediction data 206 and the prediction reference 224.

[0056] Ideally, the predicted BPU 208 can be the same as the original BPU. However, due to non-ideal prediction and reconstruction operations, the predicted BPU 208 is typically slightly different from the original BPU. To record these differences, after generating the predicted BPU 208, the encoder can subtract it from the original BPU to generate a residual BPU 210. For example, the encoder can subtract the value of the corresponding pixel of the predicted BPU 208 (e.g., grayscale value or RGB value) from the value of the pixel of the original BPU. Each pixel of the residual BPU 210 can have a residual value. As a result of this subtraction between the corresponding pixels of the original BPU and the predicted BPU 208, the predicted data 206 and the residual BPU 210 can have fewer bits than the original BPU, but they can be used to reconstruct the original BPU without noticeable quality degradation. Therefore, the original BPU is compressed,

[0057] To further compress the residual BPU 210, in the transform stage 212, the encoder can reduce the spatial redundancy of the residual BPU 210 by decomposing it into a set of two-dimensional "base patterns". Each base pattern is associated with a "transform coefficient". The base patterns can have the same size (e.g., the size of the residual BPU 210), and each base pattern can represent a changing frequency (e.g., the frequency of brightness changes) component of the residual BPU 210. None of the basic patterns can be reproduced from any combination (e.g., linear combination) of any other basic patterns. In other words, the decomposition can decompose the changes of the residual BPU 210 into the frequency domain. This decomposition is similar to the discrete Fourier transform of a function, where the base image is similar to the basic functions of the discrete Fourier transform (e.g., trigonometric functions), and the transform coefficients are similar to the coefficients associated with the basic functions.

[0058] Different transform algorithms may use different basic patterns. Various transform algorithms may be used at the transform stage 212, such as discrete cosine transform, discrete sine transform, etc. The transform at the transform stage 212 is reversible. That is, the encoder may restore the residual BPU 210 by an inverse operation of the transform (referred to as an "inverse transform"). For example, in order to restore the pixels of the residual BPU 210, the inverse transform may be to multiply the values ​​of the corresponding pixels of the basic pattern by the corresponding associated coefficients and add the products to produce a weighted sum. For video coding standards, both the encoder and the decoder may use the same transform algorithm (and therefore have the same basic pattern). Therefore, the encoder may only record the transform coefficients, from which the decoder may reconstruct the residual BPU 210 without receiving the basic pattern from the encoder. The transform coefficients may have fewer bits than the residual BPU 210, but they may be used to reconstruct the residual BPU 210 without significant quality degradation. Therefore, the residual BPU 210 is further compressed.

[0059] The encoder may further compress the transform coefficients in the quantization stage 214. During the transform process, different basic patterns may represent different frequencies of change (e.g., frequencies of brightness changes). Because the human eye is generally better at recognizing low-frequency changes, the encoder may ignore information about high-frequency changes without causing noticeable quality degradation in decoding. For example, in the quantization stage 214, the encoder may generate quantized transform coefficients 216 by dividing each transform coefficient by an integer value (referred to as a "quantization parameter") and rounding the quotient to its nearest integer. After such an operation, some transform coefficients of high-frequency basic patterns may be converted to zero, and transform coefficients of low-frequency basic patterns may be converted to smaller integers. The encoder may ignore quantized transform coefficients 216 of zero value, whereby the transform coefficients are further compressed. The quantization process is also reversible, where the quantized transform coefficients 216 may be reconstructed as transform coefficients in an inverse operation of quantization (referred to as "inverse quantization").

[0060] Because the encoder ignores the remainder of the division in the rounding operation, the quantization stage 214 may be lossy. Generally, the quantization stage 214 may contribute the most information loss in the process 200A. The greater the information loss, the fewer the number of bits required to quantize the transform coefficients 216. To achieve different levels of information loss, the encoder may use different quantization parameter values ​​or any other parameters of the quantization process.

[0061] In the binary encoding stage 226, the encoder may encode the prediction data 206 and the quantized transform coefficients 216 using a binary encoding technique, such as entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless or lossy compression algorithm. In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the encoder may encode other information in the binary encoding stage 226, such as the prediction mode used in the prediction stage 204, the parameters of the prediction operation, the transform type at the transform stage 212, the parameters of the quantization process (e.g., quantization parameters), the encoder control parameters (e.g., bit rate control parameters), etc. The encoder may use the output data of the binary encoding stage 226 to generate a video bitstream 228. In some embodiments, the video bitstream 228 may be further packaged for network transmission.

[0062] Referring to the reconstruction path of process 200A, at the inverse quantization stage 218, the encoder may perform inverse quantization on the quantized transform coefficients 216 to generate reconstructed transform coefficients. At the inverse transform stage 220, the encoder may generate a reconstructed residual BPU 222 based on the reconstructed transform coefficients. The encoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224 to be used in the next iteration of process 200A.

[0063] It should be noted that other variations of process 200A may be used to encode video sequence 202. In some embodiments, the stages of process 200A may be performed by the encoder in a different order. In some embodiments, one or more stages of process 200A may be combined into a single stage. In some embodiments, a single stage of process 200A may be divided into multiple stages. For example, transform stage 212 and quantization stage 214 may be combined into a single stage. In some embodiments, process 200A may include additional stages. In some embodiments, process 200A may omit Figure 2A one or more stages in a process.

[0064] Figure 2B A schematic diagram of another example encoding process 200B according to an embodiment of the present disclosure is shown. Process 200B can be modified from process 200A. For example, process 200B can be used by an encoder that complies with a hybrid video coding standard (e.g., H.26x series). Compared with process 200A, the forward path of process 200B also includes a mode decision stage 230, and the prediction stage 204 is divided into a spatial prediction stage 2042 and a temporal prediction stage 2044, and the reconstruction path of process 200B further includes a loop filtering stage 232 and a buffer 234.

[0065] In general, prediction techniques can be divided into two types: spatial prediction and temporal prediction. Spatial prediction (e.g., intra-image prediction or "intra-frame prediction") can use pixels from one or more already encoded adjacent BPUs in the same image to predict the current BPU. That is, the prediction reference 224 in spatial prediction may include adjacent BPUs. Spatial prediction can reduce the spatial redundancy inherent in the image. Temporal prediction (e.g., inter-image prediction or "inter-frame prediction") can use areas from one or more already encoded images to predict the current BPU. That is, the prediction reference 224 in temporal prediction may include encoded images. Temporal prediction can reduce the temporal redundancy inherent in the image.

[0066] Referring to process 200B, in the forward path, the encoder performs prediction operations in the spatial prediction stage 2042 and the temporal prediction stage 2044. For example, in the spatial prediction stage 2042, the encoder may perform intra-frame prediction. For the original BPU of the image being encoded, the prediction reference 224 may include one or more neighboring BPUs that have been encoded (in the forward path) and reconstructed (in the reconstruction path) in the same image. The encoder may generate the predicted BPU 208 by interpolating the neighboring BPUs. Interpolation techniques may include, for example, linear interpolation or interpolation, polynomial interpolation or interpolation, etc. In some embodiments, the encoder may perform interpolation at the pixel level, for example, by interpolating the value of the corresponding pixel of each pixel of the predicted BPU 208. The neighboring BPUs used for interpolation may be located in various directions relative to the original BPU, such as in the vertical direction (e.g., at the top of the original BPU), the horizontal direction (e.g., on the left side of the original BPU), the diagonal direction (e.g., at the bottom left, bottom right, top left, or top right of the original BPU), or any direction defined in the video coding standard used. For intra prediction, the prediction data 206 may include, for example, the location (eg, coordinates) of the used neighboring BPU, the size of the used neighboring BPU, interpolation parameters, the direction of the used neighboring BPU relative to the original BPU, and the like.

[0067] For another example, in the temporal prediction stage 2044, the encoder may perform inter-frame prediction. For the original BPU of the current image, the prediction reference 224 may include one or more images (referred to as "reference images") that have been encoded (in the forward path) and reconstructed (in the reconstruction path). In some embodiments, the reference image may be encoded and reconstructed by BPUs one by one. For example, the encoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a reconstructed BPU. When all reconstructed BPUs of the same image are generated, the encoder may generate a reconstructed image as a reference image. The encoder may perform a "motion estimation" operation to search for a matching area in a range (referred to as a "search window") of the reference image. The position of the search window in the reference image may be determined based on the position of the original BPU in the current image. For example, the search window may be centered at a position in the reference image having the same coordinates as the original BPU in the current image, and may extend outward by a predetermined distance. When the encoder identifies (e.g., by using a pel recursive algorithm, a block matching algorithm, etc.) an area similar to the original BPU in the search window, the encoder may determine such an area as a matching area. The matching region may have a different size than the original BPU (e.g., smaller, equal, larger, or having a different shape). Because the reference image and the current image are temporally separated on the timeline (e.g., as Figure 1 ), so the matching area can be considered to "move" to the location of the original BPU over time. The encoder can record the direction and distance of this movement as a "motion vector". Figure 1 The encoder can search for matching areas and determine the motion vector associated with each reference image. In some embodiments, the encoder can assign weights to the pixel values ​​of the matching areas of each matching reference image.

[0068] Motion estimation may be used to identify various types of motion, such as translation, rotation, scaling, etc. For inter-frame prediction, the prediction data 206 may include, for example, the location (e.g., coordinates) of the matching region, the motion vector associated with the matching region, the number of reference images, the weights associated with the reference images, etc.

[0069] To generate the predicted BPU 208, the encoder may perform a "motion compensation" operation. Motion compensation may be used to reconstruct the predicted BPU 208 based on the prediction data 206 (e.g., motion vectors) and the prediction reference 224. For example, the encoder may move a matching area of ​​a reference image according to the motion vector, where the encoder may predict the original BPU of the current image. Figure 1The encoder may move the matching region of the reference image according to the motion vectors and the average pixel value of the matching region. In some embodiments, if the encoder has assigned weights to the pixel values ​​of the matching regions of the matching reference image, the encoder may add the weighted sum of the pixel values ​​of the moving matching regions.

[0070] In some embodiments, inter-frame prediction can be unidirectional or bidirectional. Unidirectional inter-frame prediction can use one or more reference images in the same temporal direction relative to the current image. For example, Figure 1 The picture 104 in is a unidirectional inter-frame prediction picture, where the reference picture (i.e., picture 102) precedes picture 04. Bidirectional inter-frame prediction can use one or more reference pictures in both temporal directions relative to the current picture. For example, Figure 1 Picture 106 in is a bidirectional inter-predicted picture, where the reference pictures (ie, pictures 104 and 08) are relative to picture 104 in both temporal directions.

[0071] Still referring to the forward path of process 200B, after spatial prediction 2042 and temporal prediction stage 2044, at mode decision stage 230, the encoder can select a prediction mode (e.g., one of intra prediction or inter prediction) for the current iteration of process 200B. For example, the encoder can perform a rate-distortion optimization technique, in which the encoder can select a prediction mode to minimize the value of a cost function based on the bit rate of a candidate prediction mode and the distortion of a reconstructed reference image under the candidate prediction mode. Based on the selected prediction mode, the encoder can generate a corresponding prediction BPU 208 and prediction data 206.

[0072] In the reconstruction path of process 200B, if the intra-frame prediction mode has been selected in the forward path, after generating the prediction reference 224 (e.g., the current BPU that has been encoded and reconstructed in the current image), the encoder can feed the prediction reference 224 directly to the spatial prediction stage 2042 for later use (e.g., for interpolating the next BPU of the current image). If the inter-frame prediction mode has been selected in the forward path, after generating the prediction reference 224 (e.g., the current image in which all BPUs have been encoded and reconstructed), the encoder can feed the prediction reference 224 to the loop filter stage 232. At this stage, the encoder can apply a loop filter to the prediction reference 224 to reduce or eliminate distortion (e.g., blocking artifacts) introduced by the inter-frame prediction. The encoder can apply various loop filter techniques at the loop filter stage 232, such as deblocking, sampling adaptive compensation, adaptive loop filter, etc. The loop filtered reference pictures may be stored in a buffer 234 (or "decoded picture buffer") for later use (e.g., as inter-prediction reference pictures for future pictures of the video sequence 202). The encoder may store one or more reference pictures in the buffer 234 for use at the temporal prediction stage 2044. In some embodiments, the encoder may encode the parameters of the loop filter (e.g., loop filter strength) as well as the quantized transform coefficients 216, the prediction data 206, and other information at the binary encoding stage 226.

[0073] Figure 3A A schematic diagram of an exemplary decoding process 300A according to an embodiment of the present invention is shown. Process 300A may correspond to Figure 2A 300A. In some embodiments, process 300A may be similar to the reconstruction path of process 200A. The decoder may decode video bitstream 228 into video stream 304 according to process 300A. Video stream 304 may be very similar to video sequence 202. However, due to information loss during compression and decompression (e.g., Figure 2A-2B 214), typically, the video stream 304 is different from the video sequence 202. Figure 2A-2B 200A and 200B in the decoder, the decoder may perform process 300A at a basic processing unit (BPU) level for each picture encoded in the video bitstream 228. For example, the decoder may perform process 300A in an iterative manner, where the decoder may decode a basic processing unit in one iteration of process 300A. In some embodiments, the decoder may perform process 300A in parallel for regions (e.g., regions 114-118) of each picture encoded in the video bitstream 228.

[0074] like Figure 3AAs shown, the decoder may feed a portion of the video bitstream 228 associated with a basic processing unit (referred to as an "encoded BPU") of an encoded picture to a binary decoding stage 302, where the decoder may decode the portion into prediction data 206 and quantized transform coefficients 216. The decoder may feed the quantized transform coefficients 216 to an inverse quantization stage 218 and an inverse transform stage 220 to generate a reconstructed residual BPU 222. The decoder may feed the prediction data 206 to the prediction stage 204 to generate the prediction BPU 208. The decoder may add the reconstructed residual BPU 222 to the prediction BPU 208 to generate a prediction reference 224. In some embodiments, the prediction reference 224 may be stored in a buffer (e.g., a decoded picture buffer in a computer memory). The decoder may feed the prediction reference 224 to the prediction stage 204 for performing a prediction operation in the next iteration of the process 300A.

[0075] The decoder may iteratively perform process 300A to decode each coded BPU of a coded picture and generate a prediction reference 224 for the next coded BPU of the coded picture. After decoding all coded BPUs of a coded picture, the decoder may output the picture to a video stream 304 for display and continue decoding the next coded picture in the video bitstream 228.

[0076] At the binary decoding stage 302, the decoder may perform the inverse operation of the binary coding technique used by the encoder (e.g., entropy coding, variable length coding, arithmetic coding, Huffman coding, context-adaptive binary arithmetic coding, or any other lossless compression algorithm). In some embodiments, in addition to the prediction data 206 and the quantized transform coefficients 216, the decoder may decode other information at the binary decoding stage 302, such as the prediction mode, parameters of the prediction operation, the transform type, parameters of the quantization process (e.g., quantization parameters), encoder control parameters (e.g., bit rate control parameters), etc. In some embodiments, if the video bitstream 228 is transmitted over the network in the form of packets, the decoder may depacketize the video bitstream 228 before feeding it to the binary decoding stage 302.

[0077] Figure 3B A schematic diagram of another example decoding process 300B according to an embodiment of the present disclosure is shown. Process 300B can be modified from process 300A. For example, process 300B can be used by a decoder that complies with a hybrid video coding standard (e.g., H.26x series). Compared to process 300A, process 300B additionally divides the prediction stage 204 into a spatial prediction stage 2042 and a temporal prediction stage 2044, and additionally includes a loop filtering stage 232 and a buffer 234.

[0078] In process 300B, for an encoded basic processing unit (referred to as a "current BPU") of a decoded encoded image (referred to as a "current image"), the prediction data 206 decoded by the decoder from the binary decoding stage 302 may include various types of data, depending on what prediction mode the encoder used to encode the current BPU. For example, if the encoder uses intra-frame prediction to encode the current BPU, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating intra-frame prediction, parameters of the intra-frame prediction operation, etc. The parameters of the intra-frame prediction operation may include, for example, the location (e.g., coordinates) of one or more neighboring BPUs used as a reference, the size of the neighboring BPU, parameters of interpolation, the direction of the neighboring BPU relative to the original BPU, etc. For another example, if the current BPU is encoded by inter-frame prediction used by the encoder, the prediction data 206 may include a prediction mode indicator (e.g., a flag value) indicating inter-frame prediction, parameters of the inter-frame prediction operation, etc. The parameters of the inter-frame prediction operation may include, for example, the number of reference images associated with the current BPU, the weights respectively associated with the reference images, the positions (e.g., coordinates) of one or more matching regions in the corresponding reference images, one or more motion vectors respectively associated with the matching regions, and the like.

[0079] Based on the prediction mode indicator, the decoder can decide whether to perform spatial prediction (e.g., intra-frame prediction) in the spatial prediction stage 2042 or to perform temporal prediction (e.g., inter-frame prediction) in the temporal prediction stage 2044. The details of performing such spatial prediction or temporal prediction are described in Figure 2B After performing such spatial prediction or temporal prediction, the decoder may generate a predicted BPU 208, and the decoder may add the predicted BPU 208 and the reconstructed residual BPU 222 to generate a prediction reference 224, as shown in FIG. Figure 3A As described in.

[0080] In process 300B, the decoder can feed the prediction reference 224 to the spatial prediction stage 2042 or the temporal prediction stage 2044 for performing a prediction operation in the next iteration of process 300B. For example, if the current BPU is decoded using intra-frame prediction in the spatial prediction stage 2042, then after generating the prediction reference 224 (e.g., the decoded current BPU), the decoder can feed the prediction reference 224 directly to the spatial prediction stage 2042 for later use (e.g., for interpolating the next BPU of the current image). If the current BPU is decoded using inter-frame prediction in the temporal prediction stage 2044, then after generating the prediction reference 224 (e.g., the reference image in which all BPUs are decoded), the encoder can feed the prediction reference 224 to the loop filter stage 232 to reduce or eliminate distortion (e.g., blocking artifacts). The decoder can Figure 2BThe loop filter is applied to the prediction reference 224 in the manner shown. The loop filtered reference picture may be stored in a buffer 234 (e.g., a decoded picture buffer in a computer memory) for later use (e.g., as an inter-prediction reference picture for future encoded pictures of the video bitstream 228). The decoder may store one or more reference pictures in the buffer 234 for use at the temporal prediction stage 2044. In some embodiments, when the prediction mode indicator of the prediction data 206 indicates that inter-frame prediction was used to encode the current BPU, the prediction data may further include parameters of the loop filter (e.g., loop filter strength).

[0081] Figure 4 4 is a block diagram of an example apparatus 400 for encoding or decoding a video according to an embodiment of the present disclosure. Figure 4 As shown, the device 400 may include a processor 402. When the processor 402 executes the instructions described herein, the device 400 may become a special-purpose machine for video encoding or decoding. The processor 402 may be any type of circuit capable of manipulating or processing information. For example, the processor 402 may include any number of central processing units (or "CPUs"), graphics processing units (or "GPUs"), neural processing units ("NPUs"), microcontroller units ("MCUs"), optical processors, programmable logic controllers, microprocessors, digital signal processors, intellectual property (IP) cores, programmable logic arrays (PLAs), programmable array logic (PALs), general array logic (GALs), complex programmable logic devices (CPLDs), a field programmable gate array (FPGA), a system on a chip (SoC), an application-specific integrated circuit (ASIC), and the like. In some embodiments, the processor 402 may also be a group of processors grouped into a single logical component. For example, as Figure 4 As shown, processor 402 may include multiple processors, including processor 402a, processor 402b, and processor 402n.

[0082] The apparatus 400 may also include a memory 404 configured to store data (eg, instruction sets, computer codes, intermediate data, etc.). Figure 4As shown, the stored data may include program instructions (e.g., for implementing stages in process 200A, 200B, 300A, or 300B) and data for processing (e.g., video sequence 202, video bitstream 228, or video stream 304). Processor 402 may access program instructions and data for processing (e.g., via bus 410), and execute program instructions to perform operations or manipulations on data for processing. Memory 404 may include a high-speed random access storage device or a non-volatile storage device. In some embodiments, memory 404 may include any number of random access memories (RAM), read-only memories (ROM), optical disks, magnetic disks, hard drives, solid-state drives, flash drives, secure digital (SD) cards, memory sticks, compact flash (CF) cards, etc. Any combination. Memory 404 may also be a group of memories grouped into a single logical component ( Figure 4 not shown).

[0083] The bus 410 may be a communication device that transmits data between components within the apparatus 400, such as an internal bus (eg, a CPU-memory bus), an external bus (eg, a Universal Serial Bus port, a Peripheral Component Interconnect Express port), or the like.

[0084] For ease of explanation and without ambiguity, in the present disclosure, the processor 402 and other data processing circuits are collectively referred to as "data processing circuits". The data processing circuits may be implemented entirely as hardware, or as a combination of software, hardware, or firmware. In addition, the data processing circuits may be a single independent module, or may be fully or partially combined into any other component of the device 400.

[0085] The device 400 may also include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, etc.). In some embodiments, the network interface 406 may include any number of network interface controllers (NICs), radio frequency (RF) modules, transponders, transceivers, modems, routers, gateways, wired network adapters, wireless network adapters, Bluetooth adapters, infrared adapters, near field communication (“NFC”) adapters, cellular network chips, etc.

[0086] In some embodiments, the apparatus 400 may optionally further include a peripheral interface 408 to provide a connection to one or more peripheral devices. Figure 4 As shown, peripheral devices may include, but are not limited to, a cursor control device (e.g., a mouse, touch pad, or touch screen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light emitting diode display), a video input device (e.g., a camera or an input interface coupled to a video archive), and the like.

[0087] It should be noted that the video codec (e.g., a codec that performs the process 200A, 200B, 300A, or 300B) can be implemented as any combination of any software or hardware modules in the device 400. For example, some or all stages of the process 200A, 200B, 300A, or 30013 can be implemented as one or more software modules of the device 400, such as a program instance that can be loaded into the memory 404. For another example, some or all stages of the process 200A, 200B, 300A, or 300B can be implemented as one or more hardware modules of the device 400, such as a dedicated data processing circuit (e.g., FPGA, ASIC, NPU, etc.).

[0088] In the quantization and inverse quantization blocks (e.g. Figure 2A or Figure 2B quantization 214 and inverse quantization 218, Figure 3A or Figure 3B In the inverse quantization 218 of the prediction residual, the quantization parameter (QP) is used to determine the amount of quantization (and inverse quantization) applied to the prediction residual. The initial QP value for encoding a picture or slice can be signaled at a higher level, for example, using the init_qp_minus26 syntax element in the picture parameter set (PPS) and using the slice_qp_delta syntax element in the slice header. In addition, the QP value can be adapted for each CU at a local level using a delta QP value sent at the granularity of a quantization group.

[0089] According to some embodiments, a picture may be divided into a plurality of coding tree units (CTUs). The CTUs are then further divided into one or more coding units (CUs) using a quadtree (SPLIT_QT) with a nested multi-type tree structure partitioned using binary and ternary partitioning. Figure 5 is a schematic diagram showing an example of a multi-type tree partitioning mode according to some embodiments of the present disclosure. Figure 5 As shown, the partition types in the multi-type tree structure may include quadtree partition (SPLIT_QT) 501, vertical binary tree partition (SPLIT BT VER) 502, horizontal binary tree partition (SPLIT_BT_HOR) 503, vertical ternary tree partition (SPLIT_TT_VER) 504, and horizontal ternary tree partition (SPLIT_TT_HQR) 505. The multi-type leaf nodes are called coding units (CUs), which may have a square or rectangular shape.

[0090] Figure 6 is a schematic diagram of an exemplary signaling mechanism for partition division information in a quadtree with a nested multi-type tree coding tree structure according to some embodiments of the present disclosure. Figure 6In the , CTU is regarded as the root of the quadtree and is first divided by the quadtree structure. The leaf nodes of each quadtree (when large enough to allow it) are further divided by the multi-type tree structure. In the multi-type tree structure, a first flag (e.g., mtt_split_cu_flag) is signaled to indicate whether the node is further divided. When the node is further divided, a second flag (e.g., mtt_split_cu_vertical_flag) is signaled to indicate the direction of the division, and then a third flag (e.g., mtt_split_cu_binary_flag) is signaled to indicate whether the division is a binary tree division or a ternary tree division. Based on the values ​​of the second and third flags, mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, a multi-type tree division mode (MttSplitMode) of a CU can be derived. Figure 7 An exemplary Table 1 is shown, which illustrates an exemplary MTTSplitMode derivation based on multiple-type tree syntax elements according to some embodiments of the present disclosure.

[0091] A virtual pipe data unit (VPDU) is defined as a non-overlapping unit in an image. In a hardware decoder, consecutive VPDUs are processed simultaneously by multiple pipeline stages. In most pipeline stages, the VPDU size is roughly proportional to the buffer size, so it is important to maintain the VPDU size. In most hardware decoders, the VPDU size can be set to 64×64 luma samples. However, in some embodiments, ternary tree (TT) and binary tree (BT) partitioning may result in an increase in the VPDU size. Consistent with the present disclosure, in order to keep the VPDU size to 64×64 luma samples, certain specification partitioning restrictions may be applied. Figure 8 An example of disallowed TT and BT partitioning according to some embodiments of the present disclosure is shown. Figure 8 As shown, TT partitioning is not allowed for blocks with width or height, or both width and height equal to 128. For CUs of 128×N, N≤64 (e.g., width equal to 128 and height less than 128), horizontal BT is not allowed. For CUs of N×128, N≤64 (e.g., height equal to 128 and width less than 128), vertical BT is not allowed.

[0092] According to the requirements in HEVC, when part of a tree node block exceeds the bottom or right boundary of the image, the tree node block must be split until all samples of each coded CU are within the image boundary. The following splitting rules are applied in VVC draft 7:

[0093] - If part of the tree node block exceeds the bottom and right borders of the image,

[0094] - If the chunk is a QT node and the chunk size is larger than the minimum QT size, force the chunk to be split using QT split mode.

[0095] - Otherwise, the block will be forced to be split using SPLIT_BT_HOR mode.

[0096] - Otherwise, if part of the tree node block exceeds the bottom edge of the image,

[0097] - If the chunk is a QT node, and the chunk's size is greater than the minimum QT size, and the chunk's size is greater than the maximum BT size, then force the chunk to be split using QT split mode.

[0098] - Otherwise, if the chunk is a QT node and the chunk's size is greater than the minimum QT size and the chunk's size is less than or equal to the maximum BT size, force the chunk to be split using the QT split mode or the SPLIT_BT_HQR mode.

[0099] - Otherwise (the chunk is a BT node or the chunk's size is less than or equal to the minimum QT size), force the chunk to be split using SPLIT_BT_HQR mode.

[0100] - Otherwise, if part of the tree node block exceeds the right edge of the image,

[0101] - If the block is a QT node, and the block size is greater than the minimum QT size, and the block size is greater than the maximum BT size, then force the block to be split using the QT split mode,

[0102] - Otherwise, if the chunk is a QT node and the chunk's size is greater than the minimum QT size and the chunk's size is less than or equal to the maximum BT size, then force the chunk to be split using the QT split mode or the SPLIT_BT_VER mode.

[0103] - Otherwise (e.g. the chunk is a BT node or the chunk's size is less than or equal to the minimum QT size), force the chunk to be split using SPLIT_BT_VER mode.

[0104] Fig. 9 FIG. 4 shows an exemplary block partitioning on an image boundary according to some embodiments of the present disclosure. Fig. 9 As shown, for CTU 911, SPLIT_QT or SPLIT_BT_VER may be performed. For CTU 913, if SPLIT_QT is allowed, SPLIT_QT may be performed, and if SPLIT_QT is not allowed, SPLIT_BT_HOR may be performed. For CTU 915, SPLIT_QT or SPLIT_BT_HOR may be performed.

[0105] In VVC draft 7, there are two sections related to block partitioning. The first is section 6.4, which defines whether a block can be partitioned using a quadtree, binary tree, or ternary tree. The output of section 6.4 is the variables allowSplitQt, allowSplitBtHor, allowSplitBtVer, allowSplitTtHor, and allowSplitTtVer. These variables are used in Fig.12 Table 4 in Section 7.3.9.4 is used to determine whether to signal the corresponding CU level split flag (by Fig.12 4 (marked in boxes 1201-1204 in Table 4).

[0106] Section 6.4.1 of VVC draft 7 describes:

[0107] 6.4 Availability Process

[0108] 6.4.1 Allowable Quadtree Splitting Process

[0109] The inputs to this process include:

[0110] - the coding block size cbSize in luma samples,

[0111] -Multi-type tree depth mttDepth,

[0112] - the variable treeType specifies whether a single tree (SINGLE_TREE) or a dual tree is used to partition the coding tree nodes, and when using a dual tree, whether the luma (DUAL_TREE_LUMA) or chroma component (DUAL_TREE_CHROMA) is currently being processed,

[0113] - a variable modeType specifying whether intra (MODE_INTRA), IBC (MODE_IBC) and inter coding modes (MODE_TYPE_ALL) can be used, or whether only intra and IBC coding modes can be used (MODE_TYPE_INTRA), or whether only inter coding mode (MODE_TYPE_INTER) can be used for coding units inside coding tree nodes.

[0114] The output of this process is the variable allowSplitQt. The variable allowSplitQt is exported as follows:

[0115] - Set allowSplitQt to false (FALSE) if one or more of the following conditions are true (true):

[0116] -treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA, and cbSize is less than or equal to MinQtSizeY

[0117] -treeType is equal to DUAL_TREE_CHROMA, cbSize / SubWidthC is less than or equal to MinQtSizeC

[0118] -mttDepth is not equal to 0

[0119] -treeType is equal to DUAL_TREE_CHROMA, and (cbSize / SubWidthC) is less than or equal to 4

[0120] -treeType equals DUAL_TREE_CHROMA, modeType equals MODE_TYPE_INTER

[0121] Otherwise, allowSplitQt is set equal to TRUE.

[0122] Section 6.4.2 of VVC draft 7 describes:

[0123] 6.4.2 Allowable binary tree partitioning processes

[0124] The inputs to this process are:

[0125] -Binary tree partitioning mode btSplit,

[0126] - coding block width cbWidth in luma samples,

[0127] - coding block height cbHeight in luma samples,

[0128] - the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture,

[0129] -Multi-type tree depth mttDepth,

[0130] - maximum multi-type tree depth with offset maxMttDepth,

[0131] - Maximum binary tree size maxBtSize,

[0132] - minimum quadtree size minQtSize,

[0133] -Partition index partIdx

[0134] - The variable treeType specifies whether to use a single tree (SINGLE_TREE) or a dual tree to partition the coding tree nodes, and when using a dual tree, whether the luminance component (DUAL_TREE_LUMA) or the chrominance component (DUAL_TREE_CHROMA) is currently being processed,

[0135] - A variable modeType specifies whether intra (MODE_INTRA), IBC (MODE_JBC) and inter coding modes (MODE_TYPE_ALL) can be used, or whether only intra and IBC coding modes (MODE_TYPE_INTRA) can be used, or whether only inter coding mode (MODE_TYPE_INTER) can be used for coding units inside coding tree nodes.

[0136] The output of this process is the variable allowBtSplit. Fig.10 An exemplary Table 2 is shown, which shows exemplary specifications of the variables parallelTtSplit and cbSize based on btSplit, according to some embodiments of the present disclosure.

[0137] The variable allowBtSplit is derived as follows:

[0138] - allowBtSplit is set equal to FALSE if one or more of the following conditions are true:

[0139] -cbSize is less than or equal to MinBtSizeY

[0140] -cbWidth is greater than maxBtSize

[0141] -cbHeight is greater than maxBtSize

[0142] -mttDepth is greater than or equal to maxMttDepth

[0143] -treeType is equal to DUAL_TREE_CHROMA, and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 16

[0144] -treeType equals DUAL_TREE_CHROMA, and (cbWidth / SubWidthC) equals 4, btSplit equals SPLIT_BT_VER

[0145] -treeType equals DUAL_TREE_CHROMA, modeType equals MODE_TYPE_INTRA

[0146] -cbWidth*cbHeight is equal to 32, modeType is equal to MODE_TYPE_INTER

[0147] - Otherwise, if all of the following are true, allowBtSplit is set to FALSE

[0148] -btSplit equals SPLIT_BT_YER

[0149] -y0+cbHeight is greater than pic_height_in_luma_samples

[0150] - Otherwise, if all of the following conditions are true, then allowBtSplit is set equal to FALSE

[0151] -btSplit equals SPLIT_BT_VER

[0152] -cbHeight is greater than 64

[0153] -x0+cbWidth is greater than pic_width_in_luma_samples

[0154] - Otherwise, if all of the following conditions are true, then allowBtSplit is set equal to FALSE

[0155] -btSplit equals SPLIT_BT_HOR

[0156] -cbWidth is greater than 64

[0157] -y0+cbHeight is greater than pic_height_in_luma_samples

[0158] - Otherwise, if all of the following are true, allowBtSplit is set to FALSE

[0159] -x0+cbWidth is greater than pic_width_in_luma_samples

[0160] -y0+cbHeight is greater than pic_height_in_luma_samples

[0161] -cbWidth is greater than minQtSize

[0162] - Otherwise, if all of the following are true, allowBtSplit is set to FALSE

[0163] -btSplit equals SPLIT_BT_HOR

[0164] -x0+cbWidth is greater than pic_width_in_luma_samples

[0165] -y0+cbHeight is less than or equal to pic_height_in_luma_samples

[0166] - Otherwise, if all of the following are true, allowBtSplit is set to FALSE:

[0167] -mttDepth is greater than 0

[0168] -partldx is equal to 1

[0169] -MttsplitMode[x0][y0][mttDepth-1] equals parallelTtSplit

[0170] - Otherwise, if all of the following conditions are true, then allowBtSplit is set equal to FALSE

[0171] -btSplit equals SPLIT_BT_VER

[0172] -cbWidth is less than or equal to 64

[0173] -cbHeight is greater than 64

[0174] - Otherwise, if all of the following conditions are true, then allowBtSplit is set equal to FALSE

[0175] -btSplit equals SPLIT_BT_HOR

[0176] -cbWidth is greater than 64

[0177] -cbHeight is less than or equal to 64

[0178] Otherwise, allowBtSplit is set equal to TRUE.

[0179] VVC draft 7 section 6.4.3 describes:

[0180] 6.4.3 Allowed ternary tree partitioning process

[0181] The inputs to this process are:

[0182] - ternary tree partition ttSplit,

[0183] - coding block width cbWidth in luma samples,

[0184] - coding block height cbHeight in luma samples,

[0185] - the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture,

[0186] -Multi-type tree depth mttDepth

[0187] - maximum multi-type tree depth with offset maxMttDepth,

[0188] -Maximum ternary tree size maxTtSize,

[0189] - The variable treeType specifies whether to use a single tree (SINGLE_TREE) or a dual tree to divide the coding tree nodes, and when using a dual tree, whether the current processing is the luminance component (DUAL_TREE_LUMA) or the chrominance component (DUAL_TREE_CHROMA),

[0190] - A variable modeType specifies whether intra (MODE_INTRA), IBC (MODE_IBC) and inter coding modes (MODE_TYPE_ALL) can be used, or whether only intra and IBC coding modes (MODE_TYPE_INTRA) can be used, or whether only inter coding mode (MODE_TYPE_INTER) can be used for coding units inside coding tree nodes.

[0191] The output of this process is the variable allowTtSplit. Fig.11 Exemplary Table 3 is shown, which shows an exemplary specification of the variable cbSize based on ttSplit, according to some embodiments of the present disclosure.

[0192] The variable allowTtSplit is derived as follows:

[0193] - allowTtSplit is set equal to FALSE if one or more of the following conditions are true:

[0194] -cbSize is less than or equal to 2*MinTtSizeY

[0195] -cbWidth is greater than Min(64, maxTtSize)

[0196] -cbHeight is greater than Min(64, maxTtSize)

[0197] -mttDepth is greater than or equal to maxMttDeptb

[0198] -x0+cbWidth is greater than pic_width_in_luma_samples

[0199] -y0+cbHeight is greater than pic_height_in_luma_samples

[0200] -treeTpye is equal to DUAL_TREE_CHROMA, and (cbWidth / SubWidthC)*(cbHeight / SubHeightC) is less than or equal to 32

[0201] -treeType equals DUAL_TREE_CHROMA, and (cbWidth / SubWidthC)

[0202] Equal to 8, ttSplit equal to SPLIT_TT_VER

[0203] -treeType equals DUAL_TREE_CHROMA and modeType equals MODE_TYPE_INTRA

[0204] -cbWidth*cbHeight is equal to 64, modeType is equal to MODE_TYPE_INTER

[0205] Otherwise, allowTtSplit is set equal to TRUE.

[0206] Fig.12 An exemplary Table 4 is shown, which shows an exemplary section 7.3.9.4 coding tree syntax of VVC draft 7 (emphasis added in italics and shading), according to some embodiments of the present disclosure.

[0207] The variables allowSplitQt, allowSplitBtVer, allowSplitBtHor, allowSplitTtVer, and allowSplitTtHor are derived as follows:

[0208] - Invoke the allowed quadtree splitting process specified in clause 6.4.1, using the coding block size cbSize set equal to cbWidth, the current multi-type tree depth mttDepth, treeTypeCurr, and modeTypeCurr as input, and assigning the output to allowSplitQt.

[0209] - The variables minQtSize, maxBtSize, maxTtSize and maxMttDepth are derived as follows:

[0210] - If treeType is equal to DUAL_TREE_CHROMA, set minQtSize, maxBtSize, maxTtSize, and maxMttdepth equal to MinQtSizeC, MaxBtSizeC, MaxTtSizeC, and MaxMttDepthc+depthOffset, respectively.

[0211] Otherwise, minQtSize, maxBtSize, maxTtSize, and maxMttDepth are set equal to MinQtSizeY, MaxBtSizeY, MaxTtSizeY, and MaxMttDepthY + depthOffset, respectively.

[0212] - Call the allowed binary tree splitting process specified in Section 6.4.2, using the binary tree splitting mode SPLIT_BT_VER, the coding block width cbWith, the coding block height cbHeight, the position (x0, y0), the current multi-type tree depth mttDepth, the maximum multi-type tree depth with offset maxMttdepth, the maximum binary tree size maxBtSize, the minimum quadtree size minQtSize, the current partition index partIdx, treeTypeCurr and modeTypeCurr as input, and assign the output to allowSplitBtVer.

[0213] - Invoke the allowed binary tree splitting procedure specified in clause 6.4.2, using as input the binary tree splitting mode SPLIT_BT_HOR, the coding block height cbHeight, the coding block width cbWidth, the position (x0, y0), the current multi-type tree depth mttDepth, the maximum multi-type tree depth maxMttDepth with offset, the maximum binary tree size maxBtSize, the minimum quadtree size minQtSize, the current partition index partIdx, treeTypeCurr, and modeTypeCurr, and assign the output to allowSplitBtHor.

[0214] - Call the allowed ternary tree splitting process specified in clause 6.4.3 with the ternary tree splitting mode SPLIT_TT_VER, the coding block width cbWidth, the coding block height cbHeight, the position (x0, y0), the current multi-type tree depth mttDepth, the maximum multi-type tree depth with offset maxMttDepth, the maximum ternary tree size maxTtSize, treeTypeCurr and modeTypeCurr as input and the output assigned to allowSplitTtVer.

[0215] - Calls the allowed ternary tree splitting process specified in clause 6.4.3, using the ternary tree splitting mode SPLIT_TT_HOR, the coding block height cbHeight, the coding block width cbWidth, the position (x0, y0), the current multi-type tree depth mttDepth, the maximum multi-type tree depth with offset maxMttDepth, the maximum ternary tree size maxTtSize, treeTypeCurr, and modeTypeCurr as input, and assigns the output to allowSplitTtHor.

[0216] The syntax element split_cu_flag equal to 0 specifies that the coding unit is not split. The syntax element split_cu_flag equal to 1 specifies that the coding unit is split into four coding units using a quadtree partition indicated by the syntax element split_qt_flag, or into two coding units using a binary tree partition, or into three coding units using a ternary tree partition indicated by the syntax element mtt_split_cu_binary_flag. The binary or ternary tree partition can be vertical or horizontal, as indicated by the syntax element mtt_split_cu_vertical_flag.

[0217] When the syntax element split_cu_flag is not present, the value of split_cu_flag is inferred as follows:

[0218] - The value of split_cu_flag will be inferred to be equal to 1 if one or more of the following conditions are true:

[0219] -x0+cbWidth is greater than pic_width_in_luma_samples.

[0220] -y0+cbHeight is greater than pic_height_in_luma_sampless.

[0221] Otherwise, the value of split_cu_flag is inferred to be equal to 0.

[0222] The syntax element split_qt_flag specifies whether a coding unit is split into coding units with half the horizontal and vertical size.

[0223] When the syntax element split_qt_flag is not present, the following applies:

[0224] - If allowSplitQt is equal to TRUE, the value of split_qt_flag is inferred to be equal to 1.

[0225] - Otherwise, the value of split_qt_flag is inferred to be equal to 0.

[0226] The syntax element mtt_split_cu_vertical_flag equal to 0 specifies that the coding unit is split horizontally. The syntax element mtt_split_cu_vertical_flag equal to 1 specifies that the coding unit is split vertically.

[0227] When the syntax element mtt_split_cu_vertical_flag is not present, it is inferred as follows:

[0228] - If allowSplitBtHor is equal to TRUE or allowSplitTtHor is equal to TRUE, the value of mtt_split_cu_vertical_flag shall be inferred to be equal to 0.

[0229] Otherwise, the value of mtt_split_cu_vertical_flag is inferred to be equal to 1.

[0230] The syntax element mtt_split_cu_binary_flag equal to 0 specifies that the coding unit is split into three coding units using a ternary tree partition. The syntax element mtt_split_cu_binary_flag equal to 1 specifies that the coding unit is split into two coding units using a binary tree partition.

[0231] When the syntax element mtt_split_cu_binary_flag is not present, the following is inferred:

[0232] - If allowSplitBtVer is equal to FALSE and allowSplitBtHor is equal to FALSE, then the value of mtt_split_cu_binary_flag is inferred to be equal to 0.

[0233] Otherwise, if allowSplitTtVer is equal to FALSE and allowSplitTtHor is equal to FALSE, then the value of mtt_split_cu_binary_flag is inferred to be equal to 1.

[0234] Otherwise, if allowSplitBtHor is equal to TRUE and allowSplitTtVer is equal to TRUE, then the value of mtt_split_cu_binary_flag is inferred to be equal to mtt_split_cu_vertical_flag.

[0235] - Otherwise (allowSplitBtVer is equal to TRUE, allowSplitTtHor is equal to TRUE), the value of mtt_split_cu_binary_flag is inferred to be equal to mtt_split_cu_vertical_flag.

[0236] Fig.14 An exemplary Table 5 is shown according to some embodiments of the present disclosure, which shows an exemplary specification of MttSplitMode. The variable MttSplitMode[x][y][mttDepth] is derived from the value of the syntax element mtt_split_cu_vertical_flag and the value of the syntax element mtt_split_cu_binary_flag when x=x0..x0+cbWidth-1 and y=y0...y0+cbHeight-1 are defined in Table 4.

[0237] MttSplitMode[x][y][mttDepth] indicates horizontal binary tree, vertical binary tree, horizontal ternary tree, and vertical ternary tree partitioning of coding units within multiple types of trees. The array index x0, y0 specifies the position (x0, y0) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the image. Fig.13 An example of a multi-type tree splitting mode indicated by MttSplitMode according to some embodiments of the present disclosure is shown. Fig.13 As shown, the multi-type tree partitioning mode may include vertical binary tree partitioning (SPLIT_BT_VER) 1301, horizontal binary tree partitioning (SPLIT_BT_HOR) 1302, vertical fork tree partitioning (SPLIT_TT_VER) 1303 and horizontal fork tree partitioning (SPLIT_TT_HOR) 1304.

[0238] It should be noted that the CTU size, minimum block size and block size limit for quadtree, binary tree and ternary tree partitioning are signaled in the sequence parameter set or picture header.

[0239] Fig.15 An exemplary Table 6 is shown, which shows an exemplary section 7.3.2.3 Sequence Parameter Set RBSP syntax of VVC Draft 7 according to some embodiments of the present disclosure. Fig.16 An exemplary Table 7 is shown, which shows an exemplary section 7.3.2.6 Picture Header RBSP syntax of VVC Draft 7 according to some embodiments of the present disclosure.

[0240] According to some embodiments, when a portion of a tree node block exceeds the bottom or right side of a picture boundary, the tree node block is forced to be split until all samples of each coding block are within the picture boundary. However, in some cases, all tree partitioning modes are not allowed for blocks that are located at the picture boundary and contain samples that exceed the picture boundary. Fig.17 An exemplary block that does not allow QT, TT or BT partitioning at image boundaries according to some embodiments of the present disclosure is shown. Fig.17 It is shown that CTU 1701, CTU 1703, and CTU 1705 at picture boundaries do not allow QT, TT, or BT partitioning.

[0241] As a first exemplary case where all tree partitioning modes are not allowed, when the CTU size and the minimum QT size are both set to 128, all variables allowSplitQt, allowSplitBtHor, allowSplitBtVer, allowSplitTtHor, and allowSplitTtVer are set to false.

[0242] Due to the following condition variable allowSplitQt is set to false (italics for emphasis):

[0243] 6.4.1 Allowed quadtree partitioning process ...

[0244] - allowSplitQt is set equal to FALSE if one or more of the following conditions are true:

[0245] -treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA, and chSize is less than or equal to MinQtSizeY

[0246] -treeType is equal to DUAL_TREE_CHROMA, and cbSize / SubWidthC is less than or equal to MinQtSizeC ...

[0247] The variable allowSplitBtHor (italics added for emphasis) is set to false due to the following condition:

[0248] 6.4.2 Allowed binary partitioning procedures ...

[0249] - Otherwise, if all of the following are true, allowBtSplit is set to FALSE

[0250] -btSplit equals SPLIT_BT_HOR

[0251] -cbWidth is greater than 64

[0252] -y0+cbHeight is greater than pic_height_luma_samples

[0253] - Otherwise, if all of the following are true, allowBtSplit is set to FALSE

[0254] -x0+cbWidth is greater than pic_width_in_luma_samples

[0255] -y0+cbHeight is greater than pic_height_in_luma_samples

[0256] -cbWidth is greater than minQtSize

[0257] - Otherwise, if all of the following are true, allowBtSplit is set to FALSE

[0258] -btSplit equals SPLIT_BT_HOR

[0259] -x0+cbWidth is greater than pic_width_in_luma_samples

[0260] -y0+cbHeight is less than or equal to pic_height_in_luma_samples ...

[0261] The variable allowSplitBtVer is set to false (italics for emphasis) due to the following condition:

[0262] 6.4.2 Allowed binary partitioning procedures ...

[0263] - Otherwise, if all of the following are true, set allowBiSplit to FALSE

[0264] -btSplit equals SPLIT_BT_VER

[0265] -y0+cbHeight is greater than pic_height_in_luma_samples

[0266] - Otherwise, if all of the following are true, set allowBtSplit to FALSE

[0267] -btSplit equals SPLIT_BT_VER

[0268] -chHeight is greater than 64

[0269] -x0+cbWidth is greater than pic_width_in_luma_samples

[0270] The variables allowSplitTtHor and allowSplilTtVer (italics added for emphasis) are set to false due to the following:

[0271] 6.4.3 Allowed ternary tree partitioning process ...

[0272] - allowTtSplit is set equal to FALSE if one or more of the following conditions are true:

[0273] -cbSize is less than or equal to 2*MinTtSizeY

[0274] -cbWidth is greater than min(64, maxTtSize)

[0275] -cbHeight is greater than min(64, maxTtSize)

[0276] -mttDepth is greater than or equal to maxMttDepth

[0277] -x0+cbWidth is greater than pic_width_in_luma_samples

[0278] -y0+cbHeight is greater than pic_height_in_luma_samples ...

[0279] When all these variables are set to false, the CU level split flag may not be signaled. The syntax element split_cu_flag is inferred to be 1, the syntax element split_qt_flag is inferred to be 0, the syntax element mtt_split_cu_vertical_flag is inferred to be 1, and the syntax element mtt_split_cu_binary_flag is inferred to be 0. In this case, the block may be split using SPLIT_TT_VER, which may violate the restrictions of the VPDU.

[0280] As a second exemplary mode of not allowing all tree splitting, for blocks located at picture boundaries and containing samples beyond picture boundaries, all tree splitting is not allowed. When the minimum QT size is greater than the minimum CU size (syntax element log2_min_luma_codign_block_size_minus2 in the previous table) and the maximum BT / TT depth (syntax elements sps_max_mtt_hierarchy_depth_inter_slice, sps_max_mtt_hierarchy_depth_intra_slice_luma, pic_max_mtt_hierarchy_depth_inter_slice, _pic_max_mtt_hierarchy_depth_intra_slice_luma, and pic_max_mtt_hierarchy_depth_intra_slice_chroma) is equal to 0, all variables allowSplitQt, allowSplitBtHor, allowSplitBtHor, allowSplitTtHor, and allowSplitTtVer are set to false.

[0281] Fig.18 An exemplary block in which QT, BT, or BT splitting is not allowed at the image boundary according to some embodiments of the present disclosure is shown. A CTU (e.g., CTU 1801, CTU 1803, or CTU 1805) is first divided into four 64x64 blocks using a quadtree. Each 64x64 block is then not further divided. However, the portion of the block that exceeds the right and / or bottom image boundary is marked in gray, which is not allowed in the VVC design. For the blocks marked in gray, the variable allowSpiltQt is set to false due to the following conditions (emphasis in italics):

[0282] 6.4.1 Allowed quadtree partitioning process

[0283] …

[0284] - allowSplitQt is set equal to FALSE if one or more of the following conditions are true:

[0285] -treeTpye is equal to SINGLE_TREE or DUAL_TREE_LUMA, and cbSize is less than or equal to MinQtSizeY

[0286] -treeTpye is equal to DUAL_TREE_CHROMS, and cbSize / SubWidthC is less than or equal to MinQtSizeC

[0287] …

[0288] For the blocks marked in grey, the variables allowSplitBtHor and allowSplitBtVer are set to false due to the following (italics for emphasis):

[0289] 6.4.2 Allowed binary splitting procedures

[0290] …

[0291] The variable allowBtSplit is derived as follows:

[0292] - allowBtSplit is set to FALSE if one or more of the following conditions are true:

[0293] -cbSize is less than or equal to MinBtSizeY

[0294] -cbWidth is greater than maxBtSize

[0295] -cbHeight is greater than maxBtSize

[0296] Mttdepth is greater than or equal to maxMttDeptih

[0297] …

[0298] For the blocks marked in gray, the variables allowSplitTtHor and allowSplitTtVer are set to false due to the following situations (italics for emphasis):

[0299] 6.4.3 Allowed ternary splitting processes

[0300] …

[0301] The variable allowTtSplit is derived as follows:

[0302] - allowTtSplit is set to FALSE if one or more of the following conditions are true:

[0303] -cbSize is less than or equal to 2*MinTtSizeY

[0304] -cbWidth is greater than Min(64, maxTtSize)

[0305] -cbHeight is greater than Min(64, maxTtSize)

[0306] -mttDepth is greater than or equal to maxMttDepth ...

[0307] As described in the first exemplary case where all tree partitioning modes are not allowed, in the current VVC draft 7, a CU may contain samples outside the image boundary, but the CU cannot be divided again under certain conditions. In some embodiments of the present disclosure, the QT partitioning conditions in VVC can be changed. In one aspect, for a block containing samples beyond the image boundary and whose width or height is equal to N (e.g., N=128), QT partitioning is used when the minimum QT size is less than N (e.g., 128). In addition, in some embodiments, QT partitioning can also be used when the minimum QT size is equal to N (e.g., 128). In another aspect, using QT partitioning can be more straightforward than using BT or TT partitioning. Fig.19 1 is a schematic diagram showing an example of using BT and TT partitioning according to some embodiments of the present disclosure. Partitioning a block may require multiple steps. In addition, the partitioning may be different for blocks located in different locations, which may be complicated. For example, for block 1903, SPLIT_BT_HQR, SPLIT_BT_HOR, SPLIT_TT_VER, and SPLIT_TT_VER are performed in sequence.

[0308] In some embodiments, when block partitioning is not allowed, quadtree partitioning may be used and the syntax element split_qt_flag may be inferred to be 1. The syntax element split_qt_flag specifies whether to split the coding unit into coding units of half the horizontal and vertical size.

[0309] When the syntax element split_qt_fiag is not present, the following applies (italics for emphasis):

[0310] - split_qt_flag is inferred to be equal to 1 if all of the following conditions are true:

[0311] -split_cu_flag is equal to 1

[0312] -allowSplitQt, allowSplitBtHor, allowSplitBtVer, allowSplitTtHor and allowSplitTtVer equal FALSE.

[0313] Otherwise, if allowSplitQt is equal to TRUE, the value of split_qt_flag is inferred to be 1.

[0314] - Otherwise, the value of split_qt_flag is inferred to be 0.

[0315] In some embodiments, the minimum QT size constraint cannot be applied to blocks located at the image boundary. When a portion of a block exceeds the bottom or right edge of the image, a quadtree can be used to partition the block.

[0316] The allowed quadtree partitioning process is described as follows:

[0317] 6.4.1 Allowable Four-Way Split Process

[0318] The inputs to this process are:

[0319] - coding block size cbSize in luma samples,

[0320] -Multi-type tree depth mttDepth,

[0321] - The variable tree type is used to specify whether a single tree (SINGLE_TREE) or a dual tree is used to partition the coding tree nodes, and when using a dual tree, whether the luma (DUAL_TREE_LUMA) or chroma component (DUAL_TREE_CHROMA) is currently being processed.

[0322] - The variable modeType specifies whether intra (MODE_INTRA), IBC (MODE_IBC) and inter coding modes (MODE_TYPE_ALL) can be used, or whether only intra and IBC coding modes (MODE_TYPE_INTRA) can be used, or whether for coding units inside coding tree nodes only inter coding mode (MODE_TYPE_INTER) can be used.

[0323] The output of this process is the variable allowSplitQt.

[0324] The derivation of the variable allowSplitQt is as follows (italics are used for emphasis):

[0325] - allowSplitQt is set equal to true if all of the following conditions are true:

[0326] -treeTpye equals SINGLE_TREE or DUAL_TREE_LUMA

[0327] -cbSize equals 128

[0328] -MinQtSizeY equals 128

[0329] -x0+cbWidth is greater than pic_width_in_luma_samples or y0+cbHeight is greater than pic_height_in_luma_samples

[0330] - Otherwise, if all of the following are true, allowSplitOt is set equal to true:

[0331] -treeTpye equals DUAL_TREE_CHROMA

[0332] -CbSize / SubWidthC is equal to 128

[0333] -MinQtSizeC equals 128

[0334] -x0+cbWidth is greater than pic_width_in_luma_sample, or y0+cbHeight is greater than pic_height_in_luma_samples

[0335] Otherwise, if one or more of the following conditions are true, allowSplitQt is set equal to FALSE:

[0336] -treeType is equal to SINGLE_TREE or DUAL_TREE_LUMA, and cbSize is less than or equal to MinQtSizeY

[0337] -treeType is equal to DUAL_TREE_CHROMA, and cbSize / SubWidthC is less than or equal to MinQtSizeC

[0338] -mttDepth is not equal to 0

[0339] -treeType is equal to DUAL_TREE_CHROMA, and (ebSize / SubWidthC) is less than or equal to 4

[0340] -treeType equals DUAL_TREE_CHROMA, modeType equals MODE_TYPE_INTRA

[0341] - Otherwise, allowSplitQt is set equal to TRUE.

[0342] In some embodiments, bitstream conformance may be added to the syntax of the minimum QT size. It may be required that the minimum QT size be less than or equal to 64.

[0343] The syntax element sps_log2_diff_min_qt_min_cb_intra_slice_luma specifies the default difference between the base-2 logarithm of the minimum size in the luma samples of the luma leaf blocks resulting from the quadtree partitioning of the CTU, and the base-2 logarithm of the minimum coded block size in the luma samples of the luma CUs in slices where the slice_type of the SPS is equal to 2 (I). When the syntax element partition_constraints_override_enabled_flag is equal to 1, this default difference may be overridden by the syntax element pic_log2_diff_min_qt_min_cb_luma present in the PH and associated with the SPS. The value range of the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_luma is from 0 to CtbLog2SizeY - MinCbLog2SizeY, inclusive of the end values. The base-2 logarithm of the minimum size in the luma samples of the luma leaf blocks resulting from the quadtree partitioning of the CTU is derived as follows (italics are used for emphasis):

[0344] MinQtLog2SizeIntraY = sps_log2_diff_min_qt_min_cb_intra_slice_luma + MinCbLog2SizeY

[0345] VSize = Min(64, CtbSizeY)

[0346] In some embodiments, bitstream conformance d requires that the value of (1 << MinQtLog2SizeIntraY) be less than or equal to VSize.

[0347] The syntax element sps_log2_diff_min_qt_min_cb_inter_slice_luma specifies the default difference between the log2 of the minimum size in the luma samples of the luma leaf blocks resulting from the quadtree partitioning of the CTU and the log2 of the minimum luma coding block size in the luma CU luma samples in slices where the slice_type in the SPS is equal to 0 (B) or 1 (P). When the syntax element partition_constraints_override_enabled_flag is equal to 1, this default difference can be overridden by the syntax element pic_log2_diff_min_qt_min_cb_luma present in the PH in the SPS. The value range of the syntax element sps_log2_diff_min_qt_min_cb_inter_slice is from 0 to the value of CtbLog2SizeY MinCbLog2SizeY, inclusive of the end values. The log2 of the minimum size in the luma samples of the luma leaf blocks resulting from the quadtree partitioning of the CTU is derived as follows (italics are used for emphasis):

[0348] MinQtLog2SizeInterY = sps_log2_diff_min_qt_min_cb_inter_slice MinCbLog2SizeY

[0349] VSize = Min(64, CihSizeY)

[0350] In some embodiments, the consistency requirement of the bitstream is that the value of (1 << MinQtLog2SizeInterY) is less than or equal to VSize.

[0351] The syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma specifies the default difference between the log2 of the minimum size in luma samples in chroma leaf blocks resulting from the quadtree partitioning of a chroma CTU with treeType equal to DUAL_TREE_CHROMA, and the log2 of the minimum coded block size in luma samples of a chroma CU with treeType equal to DUAL_TREE_CHROMA in a slice with slice_type equal to 2(I) in the SPS. When the syntax element partition_constraints_override_enabled_flag is equal to 1, this default difference can be overridden by the syntax element pic_log2_diff_min_qt_min_cb_chroma present in the PH of the SPS. The syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma has a range of values from 0 to CtbLog2SizeY - MinCbLog2SizeY, inclusive. When absent, the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma is inferred to be equal to 0, and the log2 of the minimum size in luma samples in chroma leaf blocks resulting from the quadtree partitioning of a CTU with treeType equal to DUAL_TREE_CHROMA is derived as follows (italic emphasis):

[0352] MinQtLog2SizeIntraC = sps_log2_diff_min_qt_min_cb_intra_chroma + MinCbLog2SizeY

[0353] VSize = Min(64, CthSizeY)

[0354] In some embodiments, the bitstream conformance requirement is that the value of (1 << MinQtLog2SizeIntraC) is less than or equal to VSize.

[0355] The syntax element pic_log2_diff_min_qt_min_cb_intra_slice_luma specifies the default difference between the base-2 logarithm of the minimum size in luma samples of luma leaf blocks resulting from a quadtree partitioning of a CTU and the base-2 logarithm of the minimum coding block size in luma samples of luma CUs in slices with slice_type equal to 2(I) associated with the PH. The value of the syntax element pic_log2_diff_min_qt_min_cb_intra_slice_luma is in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of the syntax element pic_log2_diff_min_qt_min_cb_luma is inferred to be equal to the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_luma. In some embodiments, the value of the bitstream conformance requirement (1<<(pic_log2_diff_min_qt_min_cb_intra_slice_luma+MinCblog2SizeY)) is less than or equal to Min(64, CtbSizeY).

[0356] The syntax element pic_log2_diff_min_qt_min_cb_inter_slice specifies the difference between the base 2 logarithm of the minimum size of luma samples of luma leaf blocks resulting from a quadtree partitioning of the CTU and the base 2 logarithm of the minimum luma coding block size of the luma CU in the tuning point with slice_type equal to 0 (B) or 1 (P) associated with the PH. The value of the syntax element pic_log2_diff_min_qt_min_cb_inter_slice ranges from 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive, and when not present, the value of the syntax element pic_log2_diff_min_qt_min_cb_luma is inferred to be equal to the syntax element sps_log2_diff_min_qt_min_cb_inter_slice. In some embodiments, bitstream conformance requires that the value of (1<<(pic_log2_diff_min_qt_min_cb_inter_slice+MinCbLog2SizeY)) is less than or equal to Min(64, CtbSizeY).

[0357] The syntax element pic_log2_diff_min_qt_min_cb_intra_slice_chroma specifies the difference between the base 2 logarithm of the minimum size of luma samples of chroma leaf blocks resulting from quadtree partitioning of chroma CTUs with treeType equal to DUAL_TREE_CHROMA and the base 2 logarithm of the minimum coding block size of luma samples of chroma CUs with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) associated with PH. The value of the syntax element pic_log2_diff_min_qt_min_cb_intra_slice_chroma is in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of the syntax element pic_log2_diff_min_qt_min_cb_intra_slice_chroma is inferred to be equal to the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma. In some embodiments, bitstream conformance requires that the value of (1<<(pic_log2_diff_min_qt_min_cb_intra_slice_chroma+MinCbLog2SizeY)) is less than or equal to Min(64, CtbSizeY).

[0358] In some embodiments, when block partitioning is not allowed in the second example described above (where all tree partitioning modes are not allowed), quadtree partitioning may be used and the syntax element split_qt_flag may be inferred to be 1.

[0359] The syntax element split_qt_flag specifies whether the coding unit is split into coding units with half horizontal and vertical sizes. When the syntax element split_qt_flag is not present, the following applies (italic emphasis):

[0360] - split_qt_flag is inferred to be equal to 1 if all of the following conditions are true:

[0361] -split_cu_flag is equal to 1

[0362] -allowSplitQt, allowSplitBtHor, allowSplitBtVer, allowSplitTtHor and allowSplitTtVer equal FALSE.

[0363] - Otherwise, if allowSplitQt is equal to TRUE, the value of split_qt_flag is inferred to be equal to 1.

[0364] - Otherwise, the value of split_qt_flag is inferred to be equal to 0.

[0365] In some embodiments, in the above second exemplary case where all tree partitioning modes are not allowed, bitstream consistency may be added to the syntax of minimum QT size and maximum BT / TT depth.

[0366] The syntax element sps_log2_diff_min_qt_min_cb_intra_slice_luma specifies the default difference between the base 2 logarithm of the minimum size of luma samples of luma leaf blocks resulting from a quadtree partitioning of a CTU and the base 2 logarithm of the minimum coding block size of luma samples of luma CUs in slices involving slice_type equal to 2(I) of the SPS. The default difference may be overridden by the syntax element pic_log2_diff_min_qt_min_cb_luma present in the PH involving the SPS when the syntax element partition_constraints_override_enabled_flag is equal to 1. The value range of the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_luma is 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. The base 2 logarithm of the minimum size of luma samples of luma leaf blocks resulting from a quadtree partitioning of a CTU is as follows:

[0367] MinQtLog2SizeIntraY=sps_log2_diff_min_qt_min_cb_intra_slice_luma+MinCbLog2SizeY

[0368] The syntax element sps_max_mtt_hierarchy_depth_intra_slice_luma specifies the default maximum hierarchy depth of coding units resulting from multi-type tree partitioning of quadtree leaves in slices involving slice_type equal to 2(I) of the SPS. This default maximum hierarchy depth can be overridden by the syntax element pic_max_mtt_hierarchy_depth_intra_slice_luma present in the PH in the SPS when the syntax element partition_constraints_override_enabled_flag is equal to 1. The value range of the syntax element sps_max_mtt_hierarchy_depth_intra_slice_luma is 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive. In some embodiments, bitstream conformance requires that the value of (MmQtLog2SizeIntraY-sps_max_mtt_hierachy_depth_intra_slice_luma / 2) is less than or equal to MinCbLog2SizeY.

[0369] The syntax element sps_log2_diff_min_qt_min_cb_inter_slice specifies the default difference between the base 2 logarithm of the minimum size of luma samples of luma leaf blocks resulting from a quadtree partitioning of a CTU and the base 2 logarithm of the minimum luma coding block size of luma samples of luma CUs in slices involving an SPS with slice_type equal to 0 (B) or 1 (P). This default difference may be overridden by the syntax element pic_log2_diff_min_qt_min_cb_luma present in the PH involving the SPS when the syntax element partition_constraints_override_enabled_flag is equal to 1. The value range of the syntax element sps_log2_diff_min_qt_min_cb_inter_slice is 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. The base 2 logarithm of the minimum size of luma samples of luma leaf blocks resulting from a quadtree partitioning of a CTU is derived as follows:

[0370] MinQtLog2SizeInterY=sps_log2_diff_min_qt_min_cb_inter_slice+MinCbLog2SizeY

[0371] The syntax element sps_max_mtt_hierarchy_depth_inter_slice specifies the default maximum hierarchy depth for coding units resulting from multi-type tree partitioning of quadtree leaves in slices involving an SPS with slice_type equal to 0 (B) or 1 (P). When the syntax element partition_constraints_override_enabled_flag is equal to 1, the default maximum hierarchy depth can be overridden by the syntax element pic_max_mtt_hierarchy_depth_inter_slice present in the PH involving the SPS. The syntax element sps_max_mtt_hierarchy_depth_inter_slice value range is 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive. In some embodiments, bitstream conformance requires that the value of (MinQtLog2SizeInterY-sps_max_mtt_hierachy_depth_inter_slice / 2) is less than or equal to MinCbLog2SizeY.

[0372] The syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma specifies the default difference between the base-2 logarithm of the minimum size in luma samples of chroma leaf blocks resulting from quadtree partitioning of chroma CTUs with treeType equal to DUAL_TREE_CHROMA and the base-2 logarithm of the minimum coding block size in luma samples of chroma CUs with treeType equal to DUAL_TREE_CHROMA in slices involving the SPS with slice_type equal to 2(I). This default difference can be overridden by the syntax element pic_log2_diff_min_qt_min_cb_chroma present in the PH involving the SPS when the syntax element partition_constraints_override_enabled_flag is equal to 1. The value of the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma ranges from 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma is inferred to be 0. The base-2 logarithm of the minimum size among the luma samples of the chroma leaf blocks resulting from the quadtree partitioning of the CTU with treeType equal to DUAL_TREE_CHROMA is derived as follows:

[0373] MinQtLog2SizeIntraC=sps_log2_diff_min_qt_min_cb_intra_slice_chroma+MinCbLog2SizeY

[0374] The syntax element sps_max_mtt_hierarchy_depth_intra_slice_chroma specifies the default maximum hierarchy depth for chroma coding units resulting from multi-type tree partitioning of the leaves of the chroma quadtree with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) involving the SPS. This default maximum hierarchy depth may be overridden by the syntax element pic_max_mtt_hierarchy_depth_chroma present in the PH involving the SPS when the syntax element partition_constraints_override_enabled_flag is equal to 1. The syntax element sps_max_mtt_hierarchy_depth_intra_slice_chroma takes a value between 0 and 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive. When not present, the syntax element sps_max_mtt_hierarchy_depth_intra_slice_chroma is inferred to have a value of 0. In some embodiments, conformance of the bitstream requires that the value of (MinQtLog2SizeTntraC-sps_max_mtt_hierarchy_depth_intra_slice_chroma / 2) is less than or equal to MinCbLog2SizeY.

[0375] The syntax element pic_log2_diff_min_qt_min_cb_intra_slice_luma specifies the difference between the base 2 logarithm of the minimum size of luma samples of luma leaf blocks resulting from a quadtree partitioning of the CTU and the base 2 logarithm of the minimum coding block size of luma samples of luma CUs in slices with slice_type equal to 2(I) associated with the PH. The value of the syntax element pic_log2_diff_min_qt_min_cb_intra_slice_luma ranges from 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of the syntax element pic_log2_diff_min_qt_min_cb_luma is inferred to be equal to the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_luma.

[0376] The syntax element pic_max_mtt_hierarchy_depth_intra_slice_luma specifies the maximum hierarchy depth of coding units resulting from multi-type tree partitioning of quadtree leaves in slices with slice_type equal to 2(I) associated with the PH. The value of the syntax element pic_max_mtt_hierarchy_depth_intra_slice_luma is in the range of 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive. When not present, the value of the syntax element pic_max_mtt_hierarchy_depth_intra_slice_luma is inferred to be equal to the syntax element sps_max_mtt_hierarchy_depth_intra_slice_luma. In some embodiments, the value of the bitstream conformance requirement (pic_log2_diff_min_qt_min_cb_intra_slice_luma+MinCbLog2SizeY−pic_max_mtt_hierarchy_depth_intra_slice_luma / 2) is less than or equal to MinCbLog2SizeY.

[0377] The syntax element pic_log2_diff_min_qt_min_cb_inter_slice specifies the difference between the base-2 logarithm of the minimum size of luma samples of luma leaf blocks resulting from the quadtree partitioning of the CTU and the base-2 logarithm of the minimum luma coding block size of luma samples of luma CUs in slices with slice_type equal to 0 (B) or 1 (P) associated with the PH. The value of the syntax element pic_log2_diff_min_qt_min_cb_mter_slice ranges from 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of the syntax element pic_log2_diff_min_qt_min_cb_luma is inferred to be equal to the syntax element sp_log2_diff_min_qt_min_cb_inter_slice.

[0378] The syntax element pic_max_mtt_hierarchy_depth_inter_slice specifies the maximum hierarchy depth of coding units resulting from multi-type tree partitioning of quadtree leaves in slices with slice_type associated with PH equal to 0 (B) or 1 (P). The value of the syntax element pic_max_mtt_hierarchy_depth_inter_slice ranges from 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive. When not present, the value of the syntax element pic_max_mtt_hierarchy_depth_inter_slice is inferred to be equal to the syntax element sps_max_mtt_hierarchy_depth_inter_slice. In some embodiments, the value of the bitstream conformance requirement (pic_log2_diff_min_qt_min_cb_inter_slice+MinCbLog2SizeY-pic_max_mtt_hierarchy_depth_inter_slice / 2) is less than or equal to MinCbLog28izeY.

[0379] The syntax element pic_log2_diff_min_qt_min_cb_intra_slice_chroma specifies the difference between the base 2 logarithm of the minimum size of luma samples of chroma leaf blocks resulting from quadtree partitioning of chroma CTUs with treeType equal to DUAL_TREE_CHROMA and the base 2 logarithm of the minimum coding block size of luma samples of chroma CUs with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) associated with PH. The syntax element pic_log2_diff_min_qt_min_cb_intra_slice_chroma takes values ​​in the range of 0 to CtbLog2SizeY-MinCbLog2SizeY, inclusive. When not present, the value of the syntax element pic_log2_diff_min_qt_min_cb_intra_slice_chroma is inferred to be equal to the syntax element sps_log2_diff_min_qt_min_cb_intra_slice_chroma.

[0380] The syntax element pic_max_mtt_hierarchy_depth_intra_slice_chroma specifies the maximum hierarchy depth of chroma coding units resulting from multi-type tree partitioning with chroma quadtree leaves with treeType equal to DUAL_TREE_CHROMA in slices with slice_type equal to 2(I) associated with the PH. The value of the syntax element pic_max_mtt_hierarchy_depth_intra_slice_chroma ranges from 0 to 2*(CtbLog2SizeY-MinCbLog2SizeY), inclusive. When not present, the value of the syntax element pic_max_mtt_hierarchy_depth_intra_slice_chroma is inferred to be equal to the syntax element sps_max_mtt_hierarchy_depth_intra_slice_chroma. In some embodiments, conformance of the bitstream requires that (pic_log2_diff_min_qt_min_cb_intra_slice_chroma+MinCbLog2SizeY-pic_max_mtt_hierarchy_depth_intra_slice_chroma / 2) is less than or equal to MinCbLog2SizeY.

[0381] Fig. 20 2000 according to some embodiments of the present disclosure. The method 2000 may be performed by an encoder (e.g., by Figure 2A Process 200A or Figure 2B 200B), decoder (for example, by Figure 3A Process 300A or Figure 3B 300B) or by a device (e.g., Figure 4 For example, a processor (e.g., Figure 4 The method 2000 may be performed by a processor 402 of the computer system. In some embodiments, the method 2000 may be implemented by a computer program product contained in a computer-readable medium, wherein the computer program product includes a computer (e.g., Figure 4 Computer executable instructions, such as program codes, executed by device 400).

[0382] In step 2001, it may be determined whether a coding block includes samples outside of an image boundary. In some embodiments, the image boundary may be a bottom image boundary or a right image boundary. As an example of a coding block including samples outside of an image boundary, Fig.17, coded block 1701 exceeds the right image boundary of image 1700 , coded block 1705 exceeds the bottom image boundary of image 1700 , and coded block 1703 exceeds both the bottom and right image boundaries of image 1700 .

[0383] In step 2003, in response to the coding block being determined to include samples outside the picture boundary, the coding block can be split using the QT mode. In some embodiments, in response to the coding block being determined to include samples outside the picture boundary, the method 2000 can determine that the coding block is not allowed to be split using the BT mode and the TT mode. For example, the variables allowSplitBtHor, allowSplitBtVer, allowSplitTtHor, and allowSplitTtVer can be determined to be equal to FALSE.

[0384] In some embodiments, in response to a coding block being determined to include samples outside a picture boundary, method 2000 may determine that the coding block is to be segmented using a QT mode, regardless of whether a QT flag is present in a bitstream including the coding block. The QT flag indicates whether the coding block is segmented using a QT mode. For example, when the syntax element split_qt_flag is not present, if the coding block includes samples outside a picture boundary and the variables allowSplitQt, allowSplitBtHor, allowSplitBtVer, allowsplitTtHor, and allowSplitTtVer are equal to FALSE or allowSplitQt is equal to TRUE, then the value of split_qt_flag is inferred to be equal to 1.

[0385] In some embodiments, in response to determining that the coding block includes samples outside of a picture boundary, method 2000 may determine that the coding block is allowed to be segmented using the QT mode, regardless of a preset constraint on the minimum block size to which the QT mode is allowed to be applied. For example, the minimum QT size constraint cannot be applied to coding blocks located at a picture boundary.

[0386] In some embodiments, the preset constraints may include bitstream consistency of the coding block. The bitstream consistency may set the minimum block size that allows the QT mode to be applied. For example, the minimum block size may be set to be less than or equal to 64. The bitstream consistency may also set the maximum BT depth or the maximum TT depth.

[0387] In some embodiments, method 2000 may include determining that the coding block is to be split. For example, a syntax element split_cu_flag may be used to indicate whether the coding block is to be split. When the syntax element split_cu_flag is not present, it may be inferred that it is equal to 1, which indicates that the coding block is to be split.

[0388] It should be understood that an embodiment of the present disclosure may be combined with another embodiment or some other embodiments.

[0389] The embodiments may be further described using the following terms:

[0390] 1. A video processing method, comprising:

[0391] determining whether the coding block includes samples outside the picture boundary; and

[0392] In response to determining that the coding block includes samples outside a picture boundary, quadtree segmentation of the coding block is performed regardless of a value of a first parameter, wherein the first parameter indicates whether the quadtree is allowed to be used to segment the coding block.

[0393] 2. The method according to clause 1, further comprising:

[0394] determining a value of a first flag of the coding block, the first flag indicating whether the coding block is divided into a plurality of sub-blocks; and

[0395] Determine the values ​​of the second parameter, the third parameter, the fourth parameter and the fifth parameter of the coding block, wherein the second parameter, the third parameter, the fourth parameter and the fifth parameter respectively indicate whether it is allowed to use a binary horizontal tree, a binary vertical tree, a ternary horizontal tree, and a ternary vertical tree to split the coding block.

[0396] 3. The method according to clause 2, further comprising:

[0397] In response to the value of the first flag being equal to 1, and the values ​​of the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter being equal to 0, the value of the second flag of the coding block is set to 1, and the second flag indicates whether to use the quadtree to split the coding block.

[0398] 4. The method according to clause 2, further comprising:

[0399] In response to the value of the first parameter being equal to 1, setting the value of a second flag of the coding block to 1, the second flag indicating whether to use the quadtree to split the coding block.

[0400] 5. A method according to any one of clauses 2 to 4, further comprising:

[0401] When it is determined that the coding block includes samples outside the picture boundary, the value of the first flag is set to 1.

[0402] 6. A video processing device, comprising:

[0403] at least one memory for storing instructions, and

[0404] At least one processor is configured to execute the instructions to cause the apparatus to perform the following operations:

[0405] determining whether the coding block includes samples outside the picture boundary; and

[0406] In response to determining that the coding block includes samples outside a picture boundary, quadtree segmentation of the coding block is performed regardless of a value of a first parameter, wherein the first parameter indicates whether the quadtree is allowed to be used to segment the coding block.

[0407] 7. An apparatus as described in clause 6, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform:

[0408] determining a value of a first flag of the coding block, the first flag indicating whether the coding block is divided into a plurality of sub-blocks; and

[0409] Determine the values ​​of the second parameter, the third parameter, the fourth parameter and the fifth parameter of the coding block, wherein the second parameter, the third parameter, the fourth parameter and the fifth parameter respectively indicate whether it is allowed to use a binary horizontal tree, a binary vertical tree, a ternary horizontal tree, and a ternary vertical tree to split the coding block.

[0410] 8. An apparatus as described in clause 7, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform:

[0411] In response to the value of the first flag being equal to 1, and the values ​​of the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter being equal to 0, setting the value of the second flag of the coding block to 1, the second flag indicating whether to use a quadtree to split the coding block,

[0412] 9. An apparatus as described in clause 7, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform:

[0413] In response to the value of the first parameter being equal to 1, setting the value of a second flag of the coding block to 1, the second flag indicating whether to use the quadtree to partition the coding block.

[0414] 10. An apparatus as described in any of clauses 7 to 9, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform:

[0415] When the coding block is determined to include samples outside the picture boundary, the value of the first flag is set to 1.

[0416] 11. A non-transitory computer-readable storage medium storing a set of instructions, wherein the set of instructions can be executed by one or more processing devices to cause a video processing apparatus to perform the following operations:

[0417] Determine whether the coding block includes samples outside the picture boundary, and

[0418] In response to the coding block being determined to include samples outside a picture boundary, quadtree segmentation of the coding block is performed regardless of a value of a first parameter, wherein the first parameter indicates whether the quadtree is allowed to be used to segment the coding block.

[0419] 12. The non-transitory computer-readable storage medium of clause 11, wherein the set of instructions is executable by the one or more processing devices to cause the video processing apparatus to perform:

[0420] determining a value of a first flag of the coding block, the first flag indicating whether the coding block is divided into a plurality of sub-blocks; and

[0421] Determine the values ​​of the second parameter, the third parameter, the fourth parameter and the fifth parameter of the coding block, wherein the second parameter, the third parameter, the fourth parameter and the fifth parameter respectively indicate whether it is allowed to use a binary horizontal tree, a binary vertical tree, a ternary horizontal tree, and a ternary vertical tree to split the coding block.

[0422] 13. The non-transitory computer-readable storage medium of clause 12, wherein the set of instructions is executable by the one or more processing devices to cause the video processing apparatus to perform:

[0423] In response to the value of the first flag being equal to 1, and the values ​​of the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter being equal to 0, the value of the second flag of the coding block is set to 1, and the second flag indicates whether to use a quadtree to split the coding block.

[0424] 14. The non-transitory computer-readable storage medium of clause 12, wherein the set of instructions is executable by the one or more processing devices to cause the video processing apparatus to perform:

[0425] In response to the value of the first parameter being equal to 1, setting the value of a second flag of the coding block to 1, the second flag indicating whether to use the quadtree to partition the coding block.

[0426] 15. The non-transitory computer-readable storage medium of any of clauses 12-14, wherein in response to the coding block being determined to include samples outside a picture boundary, segmenting the coding block using the QT mode comprises:

[0427] When the coding block is determined to include samples outside the picture boundary, the value of the first flag is set to 1.

[0428] 16. A video processing method, comprising:

[0429] determining whether the coding block includes samples outside the picture boundary;

[0430] In response to the coding block being determined to include samples outside a picture boundary, the coding block is partitioned using a quadtree (QT) mode.

[0431] 17. The method according to clause 16, further comprising:

[0432] In response to the coding block being determined to include samples outside a picture boundary, it is determined that splitting the coding block using a binary tree (BT) mode and a ternary tree (TT) mode is not allowed.

[0433] 18. A method according to any of clauses 16 and 17, wherein in response to the coding block being determined to include samples outside a picture boundary, partitioning the coding block using a quadtree (QT) mode comprises:

[0434] Regardless of whether a QT flag is present in a bitstream including the coding block, the coding block is segmented using the QT mode, the QT flag indicating whether the coding block is to be segmented using the QT mode.

[0435] 19. A method as described in clause 16, wherein in response to the coding block being determined to include samples outside a picture boundary, partitioning the coding block using a quadtree (QT) mode comprises:

[0436] The coding blocks are split using the QT mode regardless of a preset constraint on a minimum block size that allows the application of the QT mode.

[0437] 20. A method as claimed in clause 19, wherein:

[0438] The preset constraints include a bitstream consistency associated with the coding block, the bitstream consistency setting a minimum block size that allows the QT mode to be applied.

[0439] 21. The method of clause 20, wherein the minimum block size is set to be less than or equal to 64.

[0440] 22. A method according to any of clauses 20 and 21, wherein the bitstream conformance sets a maximum BT depth or a maximum TT depth.

[0441] 23. A method according to any of clauses 16-22, wherein the image border is a bottom image border or a right image border.

[0442] 24. A video processing device, comprising:

[0443] at least one memory for storing instructions; and

[0444] at least one processor configured to execute the instructions to cause the apparatus to perform:

[0445] determining whether the coding block includes samples outside the picture boundary;

[0446] In response to the coding block being determined to include samples outside a picture boundary, the coding block is partitioned using a quadtree (QT) mode.

[0447] 25. An apparatus according to clause 24, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform:

[0448] In response to the coding block being determined to include samples outside a picture boundary, it is determined that splitting the coding block using a binary tree (BT) mode and a ternary tree (TT) mode is not allowed.

[0449] 26. An apparatus according to any of clauses 24 and 25, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform;

[0450] Regardless of whether a QT flag is present in a bitstream including the coding block, the coding block is segmented using the QT mode, the QT flag indicating whether the coding block is to be segmented using the QT mode.

[0451] 27. An apparatus as described in clause 24, wherein the at least one processor is configured to execute the instructions to cause the apparatus to perform:

[0452] The coding blocks are segmented using the QT mode regardless of a preset constraint on a minimum block size that allows the application of the QT mode.

[0453] 28. The apparatus according to clause 27, wherein

[0454] The preset constraints include a bitstream consistency associated with the coding block, the bitstream consistency setting a minimum block size that allows the QT mode to be applied.

[0455] 29. An apparatus as recited in clause 28, wherein the minimum block size is set to be less than or equal to 64.

[0456] 30. Apparatus according to any of clauses 28 and 29, wherein the bitstream conformance sets a maximum BT depth or a maximum TT depth.

[0457] 31. Apparatus according to any of clauses 24 to 30, wherein the image border is a bottom image border or a right image border.

[0458] 32. A non-transitory computer-readable storage medium storing a set of instructions, wherein the set of instructions can be executed by one or more processing devices to cause a video processing apparatus to perform the following operations:

[0459] determining whether the coding block includes samples outside the picture boundary;

[0460] In response to the coding block being determined to include samples outside a picture boundary, the coding block is partitioned using a quadtree (QT) mode.

[0461] 33. The non-transitory computer-readable storage medium of clause 32, wherein the set of instructions is executable by the one or more processing devices to cause the video processing apparatus to perform:

[0462] In response to the coding block being determined to include samples outside a picture boundary, it is determined that a binary tree (BT) mode and a ternary tree (TT) mode are not allowed to be used to split the coding block.

[0463] 34. The non-transitory computer-readable storage medium of any of clauses 32 and 33, wherein the set of instructions is executable by the one or more processing devices to cause the video processing apparatus to perform:

[0464] Regardless of whether a QT flag is present in a bitstream including a coding block, the coding block is segmented using the QT mode, the QT flag indicating whether the coding block is to be segmented using the QT mode.

[0465] 35. The non-transitory computer-readable storage medium of clause 32, wherein the set of instructions is executable by the one or more processing devices to cause the video processing apparatus to perform:

[0466] The coding blocks are segmented using the QT mode regardless of the preset constraint of the minimum block size that allows the application of the QT mode.

[0467] 36. A non-transitory computer-readable storage medium according to clause 35, wherein: the preset constraint includes a bitstream consistency associated with the coding block, the bitstream consistency setting a minimum block size that allows the QT mode to be applied.

[0468] 37. A non-lateral computer-readable storage medium as described in clause 36, wherein the minimum block size is set to be less than or equal to 64.

[0469] 38. The non-transitory computer-readable storage medium of any of clauses 36 and 37, wherein the bitstream conformance sets a maximum BT depth or a maximum TT depth.

[0470] 39. The non-transitory computer-readable storage medium of any one of clauses 32 to 38, wherein the image border is a bottom image border or a right image border.

[0471] In some embodiments, a non-transitory computer-readable storage medium including instructions is also provided, and the instructions can be executed by a device (such as the disclosed encoder and decoder) for performing the above method. Common forms of non-transitory media include, for example, floppy disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with hole patterns, RAM, PROM and EPROM, FLASH-EPROM or any other flash memory, NVRAM, cache, registers, any other memory chip or cartridge storage, and networked versions thereof. The device may include one or more processors (CPU), input / output interfaces, network interfaces, and / or memories.

[0472] It should be noted that relational terms, such as "first" and "second", are used herein only to distinguish an entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. In addition, the words "comprise", "have", "include" and "includes" and other similar forms are equivalent in meaning and are open-ended in that one or more items following any of these words do not mean an exhaustive list of the item or items, or are limited to the listed items or items.

[0473] As used herein, unless otherwise specifically stated, the term "or" includes all possible combinations unless not feasible. For example, if it is stated that a database may include A or B, then unless otherwise explicitly stated or not feasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then unless otherwise explicitly stated or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A, B, and C.

[0474] It should be understood that the above embodiments can be implemented by hardware, or software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above-mentioned computer-readable medium. The software can execute the disclosed method when executed by a processor. The computing unit and other functional units described in this disclosure can be implemented by hardware, or software, or a combination of hardware and software. Those of ordinary skill in the art will also understand that the above-mentioned multiple modules / units can be combined into one module / unit, and each of the above-mentioned modules / units can be further divided into multiple sub-modules / sub-units.

[0475] In the foregoing description, embodiments have been described with reference to many specific details, which may vary from implementation to implementation. Certain modifications and changes may be made to the described embodiments. Other embodiments will be apparent to those skilled in the art by considering the description and practice of the invention disclosed herein. This description and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims herein. The sequence of steps shown in the accompanying drawings is for illustrative purposes only and is not intended to be limited to any particular sequence of steps. Therefore, it will be appreciated by those skilled in the art that these steps may be performed in different orders while implementing the same method.

[0476] In the drawings and the specification, exemplary embodiments have been disclosed. However, many changes and modifications may be made to these embodiments. Therefore, although specific terms are employed, they are used only in a general and descriptive sense and not for the purpose of limitation.

Claims

1. A video processing method, comprising: Determine whether the coding block includes samples outside the picture boundary, and In response to the coding block being determined to include samples outside a picture boundary, performing quadtree partitioning of the coding block, comprising: Determining a value of a first flag of the coding block, the first flag indicating whether the coding block is divided into a plurality of sub-blocks; Determine values ​​of a first parameter, a second parameter, a third parameter, a fourth parameter, and a fifth parameter of the coding block, wherein the first parameter, the second parameter, the third parameter, the fourth parameter, and the fifth parameter respectively indicate whether it is allowed to use a quadtree, a binary horizontal tree, a binary vertical tree, a ternary horizontal tree, and a ternary vertical tree to split the coding block; as well as In response to the value of the first flag being equal to 1, and the values ​​of the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter being equal to 0, the value of the second flag of the coding block is set to 1, and the second flag indicates whether to use a quadtree to split the coding block.

2. The method according to claim 1, further comprising: In response to the value of the first parameter being equal to 1, determining the value of the second flag of the coding block to be 1.

3. The method according to claim 1, further comprising: When the coding block is determined to include samples outside the picture boundary, the value of the first flag is set to 1.

4. The method according to claim 1, further comprising: The value of the first parameter is determined based on one or more of the following: The size of the coding block; Multiple types of tree depth; a sixth parameter indicating whether a single tree or a dual tree is used to partition the coding block; or A seventh parameter indicates whether one or more of an intra mode, an inter mode, and an intra block copy (IBC) mode is allowed to encode the coding block.

5. The method according to claim 1, characterized in that In response to the coding block being determined to include samples outside a picture boundary, performing quadtree partitioning of the coding block: Quadtree partitioning of the coding block is performed regardless of a bitstream conformance associated with the coding block.

6. The method according to claim 5, characterized in that The bitstream consistency sets a minimum block size that allows the quadtree partitioning to be applied.

7. The method according to claim 6, wherein: The minimum block size is set to be less than or equal to 64.

8. The method according to claim 5, characterized in that The bitstream consistency sets the maximum binary tree depth or the maximum ternary tree depth.

9. A video processing device, characterized in that: include: at least one memory, the at least one memory being configured to store instructions; as well as at least one processor configured to execute the instructions so that the apparatus performs the following operations: determining whether the coding block includes samples outside the picture boundary; as well as In response to the coding block being determined to include samples outside a picture boundary, performing quadtree partitioning of the coding block, comprising: Determining a value of a first flag of the coding block, the first flag indicating whether the coding block is divided into a plurality of sub-blocks; as well as Determine values ​​of a second parameter, a third parameter, a fourth parameter, and a fifth parameter of the coding block, wherein the second parameter, the third parameter, the fourth parameter, and the fifth parameter respectively indicate whether it is allowed to use a binary horizontal tree, a binary vertical tree, a ternary horizontal tree, and a ternary vertical tree to split the coding block; In response to the value of the first flag being equal to 1, and the values ​​of the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter being equal to 0, the value of the second flag of the coding block is set to 1, and the second flag indicates whether to use a quadtree to split the coding block.

10. The device according to claim 9, wherein: The at least one processor is configured to execute the instructions to cause the apparatus to perform: In response to the value of the first parameter being equal to 1, determining the value of the second flag of the coding block to be 1.

11. The device according to claim 9, wherein: The at least one processor is configured to execute the instructions to cause the apparatus to perform: When the coding block is determined to include samples outside the picture boundary, the value of the first flag is set to 1.

12. A non-transitory computer-readable storage medium storing a bitstream, wherein the bitstream can be generated when one or more processing devices execute an instruction set, wherein the instruction set can be executed by the one or more processing devices to cause a video processing device to perform the following operations: Determine whether the coding block includes samples outside the picture boundary, and In response to the coding block being determined to include samples outside a picture boundary, performing quadtree partitioning of the coding block, comprising: Determining a value of a first flag of the coding block, the first flag indicating whether the coding block is divided into a plurality of sub-blocks; as well as Determine the values ​​of the second parameter, the third parameter, the fourth parameter and the fifth parameter of the coding block, wherein the second parameter, the third parameter, the fourth parameter and the fifth parameter respectively indicate whether it is allowed to use a binary horizontal tree, a binary vertical tree, a ternary horizontal tree, and a ternary vertical tree to split the coding block. In response to the value of the first flag being equal to 1, and the values ​​of the first parameter, the second parameter, the third parameter, the fourth parameter and the fifth parameter being equal to 0, set the value of the second flag of the coding block to 1, and the second flag indicates whether to use a quadtree to split the coding block.

13. The non-transitory computer-readable storage medium of claim 12, wherein the set of instructions is executed by the one or more processing devices to cause the video processing apparatus to perform: In response to the value of the first parameter being equal to 1, determining the value of the second flag of the coding block to be 1.

14. The non-transitory computer-readable storage medium according to claim 12, wherein the set of instructions can be executed by one or more processing devices to cause the video processing apparatus to further perform the following operations: When the coding block is determined to include samples outside the picture boundary, the value of the first flag is set to 1.

Images