Coding subblocks by generative intra prediction in video coding

Abstract

A video codec may be configured to code video data by performing partitioning of coding units (CUs) of the video data using intra sub-partitioning (ISP) to form prediction block sets. A video codec may group a plurality of prediction blocks from a prediction block set into a first prediction block group (PBG). The video codec may reconstruct samples of prediction blocks contained in the first PBG independently of samples of other prediction blocks contained in the first PBG.

Description

[0001] This application is a divisional application of a Chinese patent application filed on February 26, 2020, with application number 202080016068.2 and invention name “Encoding of sub-blocks by generalized intra-frame prediction in video coding”.

[0002] This application claims priority to U.S. Patent Application No. 16 / 800,467, filed on February 25, 2020, which claims priority to U.S. Provisional Patent Application No. 62 / 812,078, filed on February 28, 2019, U.S. Provisional Patent Application No. 62 / 817,474, filed on March 12, 2019, and U.S. Provisional Patent Application No. 62 / 862,936, filed on June 18, 2019, the entire contents of each of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to video encoding and video decoding. Background Art

[0004] Digital video capabilities can be integrated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), portable or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio telephones, so-called "smart phones", video teleconferencing devices, video streaming devices, etc. Digital video devices implement video codec techniques, such as those described in the standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Codec (AVC), ITU-T H.265 / High Efficiency Video Codec (HEVC), and extensions of such standards. By implementing such video codec techniques, video devices can more efficiently send, receive, encode, decode and / or store digital video information.

[0005] Video coding (e.g., video encoding and / or video decoding) generally involves predicting a block of video data from a block of coded video data in the same picture (e.g., intra-frame prediction) or a block of coded video data in a different picture (e.g., inter-frame prediction). Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) may be partitioned into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. For video blocks in an intra-coded (I) slice of a picture, spatial prediction relative to reference samples in neighboring blocks in the same picture may be used for encoding. For video blocks in an inter-coded (P or B) slice of a picture, spatial prediction relative to reference samples in neighboring blocks in the same picture or temporal prediction relative to reference samples in other reference pictures may be used. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. Summary of the invention

[0006] In general, the present disclosure describes techniques related to encoding multiple prediction blocks into prediction block groups using intra sub-partition (ISP) encoding. Specifically, a video codec (e.g., a video encoder and / or a video decoder) can group multiple prediction blocks (e.g., multiple sub-blocks of a coding unit) into prediction block groups. For example, a single prediction block group can include more than one prediction block or sub-block of a coding unit. In one example, a first prediction block group can include a first sub-block of a coding unit and a second sub-block of the coding unit.

[0007] In some examples, the video codec may reconstruct samples of a prediction block included in the first prediction block group independently of reconstruction of samples of other prediction blocks included in the same prediction block group. For example, the video codec may reconstruct samples of one prediction block before reconstructing samples of another prediction block included in the same prediction block group, without reconstructing samples of another prediction block included in the same prediction block group, or in parallel with reconstructing samples of another prediction block included in the same prediction block group. In some instances, the prediction block grouping techniques of the present disclosure may be implemented when an instance of coding using intra-frame sub-division includes partitioning of a coding unit (such as vertical partitioning, a combination of horizontal partitioning and vertical partitioning, etc.). In addition, the prediction block grouping techniques of the present disclosure may be implemented when the prediction block has a relatively narrow dimension (e.g., a narrow height, a narrow width, etc.) such that the dimension of the prediction block meets a predefined threshold for prediction block grouping. One or more techniques described herein for grouping prediction blocks into prediction block groups may be applied to any of the existing video codecs, such as the HEVC (High Efficiency Video Coding) or H.266 / Versatile Video Coding (VVC) standards, or may be used in any future video coding standards.

[0008] According to an example, a method for encoding video data is disclosed. The method includes: using intra-frame sub-partitioning (ISP) to perform partitioning of a coding unit (CU) of video data to form a prediction block set, the prediction block including at least a first prediction block and a second prediction block; grouping a plurality of prediction blocks from the prediction block set into a first prediction block group (PBG); and reconstructing samples of a prediction block included in a first PBG independently of samples of other prediction blocks included in the first PBG.

[0009] According to another example, a device for encoding video data is disclosed. The device includes a memory configured to store video data and one or more processors, the one or more processors are implemented in a circuit and configured to perform partitioning of a coding unit (CU) of the video data using intra-frame sub-partitioning (ISP) to form a prediction block set, the prediction block including at least a first prediction block and a second prediction block; grouping multiple prediction blocks from the prediction block set into a first prediction block group (PBG); and reconstructing samples of the prediction blocks contained in the first PBG independently of samples of other prediction blocks contained in the first PBG.

[0010] According to another example, a non-transitory computer-readable storage medium is disclosed. The non-transitory computer-readable storage medium has instructions stored thereon, which, when executed, cause one or more processors to: perform partitioning of a coding unit (CU) of video data using intra-frame sub-partitioning (ISP) to form a prediction block set, the prediction block including at least a first prediction block and a second prediction block; group multiple prediction blocks from the prediction block set into a first prediction block group (PBG); and reconstruct samples of the prediction blocks contained in the first PBG independently of samples of other prediction blocks contained in the first PBG.

[0011] According to another example, an apparatus for encoding video data is disclosed. The apparatus includes a component for performing partitioning of a coding unit (CU) of video data using intra sub-partitioning (ISP) to form a prediction block set, the prediction block including at least a first prediction block and a second prediction block; grouping a plurality of prediction blocks from the prediction block set into a first prediction block group (PBG); and reconstructing samples of a prediction block included in the first PBG independently of samples of other prediction blocks included in the first PBG.

[0012] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure.

[0014] Figure 2 is a diagram illustrating an exemplary intra prediction mode.

[0015] Figure 3 is a block diagram illustrating an example of an 8×4 rectangular block.

[0016] Figures 4A-4C is a block diagram providing an illustration of a mode mapping process(es) for modes outside of a diagonal direction range.

[0017] Figure 5 is a block diagram illustrating the wide angle employed in VTM2.

[0018] Figure 6 is a block diagram illustrating the wider perspective adopted in VTM3.

[0019] Figure 7 It is a table that specifies the mapping table between predModeIntra and the angle parameter intraPredAngle in VTM3.

[0020] Figure 8 is a block diagram illustrating an exemplary partitioning of a block for intra-frame sub-partition coding.

[0021] Fig. 9 is a block diagram illustrating an exemplary partitioning of a block for intra-frame sub-partition coding.

[0022] Fig.10 is a block diagram illustrating multiple reference line prediction.

[0023] Fig.11 is a conceptual diagram illustrating an example of how a coding block may be divided into four horizontal plane prediction blocks.

[0024] Fig.12 is a conceptual diagram illustrating an example of how a coding block may be divided into four vertical plane prediction blocks.

[0025] Fig.13 is a conceptual diagram illustrating an example of such a planar prediction block structure.

[0026] Fig.14 is a block diagram illustrating how a coding block is divided into four prediction blocks.

[0027] Fig.15 is a block diagram illustrating an example of coding a 4×N coding block using intra sub-partitioning and vertical partitioning.

[0028] Fig.16 is a conceptual diagram illustrating an example of dividing an M×N coding block into four prediction blocks.

[0029] Fig.17 is a conceptual diagram illustrating an example of allowing only vertical intra mode prediction for a 4×N coding block coded using intra sub-partitioning coding (ISP) and vertical partitioning.

[0030] Fig.18 is a conceptual diagram illustrating an example of 8×N (N>4) coding blocks that are coded using ISP and divided vertically.

[0031] Fig.19A and 19B is a conceptual diagram illustrating an exemplary quadtree binary tree (QTBT) structure and a corresponding coding tree unit (CTU).

[0032] Fig. 20 is a block diagram illustrating an example video encoder that may perform the techniques of this disclosure.

[0033] Fig.21 is a block diagram illustrating an example video decoder that may perform the techniques of this disclosure.

[0034] Fig. 22 is a flow chart illustrating an exemplary method for encoding a current block.

[0035] Fig.23 is a flow chart illustrating an exemplary method for decoding a current block.

[0036] Fig.24 is a flow chart illustrating an exemplary method for performing prediction block grouping on a current block of video data. DETAILED DESCRIPTION

[0037] In general, the present disclosure describes techniques related to improving the encoding and / or decoding of prediction blocks using prediction block groups in intra-frame sub-partitioning (ISP) coding. These techniques can be applied to current or future video coding standards, including the Universal Video Coding (VVC) standard currently under development.

[0038] In some examples, the video codec may encode a block of video data using ISP coding. For example, the video codec may partition a block of video data into sub-blocks (e.g., prediction blocks). In some examples, the sub-blocks may have a relatively narrow size in one or more dimensions. In an illustrative example, a coding unit (CU) (such as a coding block) may have dimensions of width (W) × height (H). In one example, W of a CU may be equal to 4 and H may be equal to N>0. In an illustrative example, the video codec may encode a 4×N CU using ISP coding. For example, the video codec may partition a CU using vertical partitioning or horizontal partitioning. In another example, the video codec may use a combination of vertical partitioning and horizontal partitioning. In this case, the video codec may partition a first one or more parts of a CU using vertical partitioning, and partition a second one or more parts of the same CU using horizontal partitioning.

[0039] In an example involving vertical partitioning, the resulting sub-block may have a size of 1×N. In addition, N may be greater than 8 (e.g., H is greater than 8). In such an example, the video codec may utilize row-by-row storage of samples. When the video codec uses row-by-row storage of samples, narrow sub-blocks such as those with dimensions below a predefined threshold may cause the video codec to perform an inefficient and time-consuming memory access process. For example, in such an instance, when performing the memory access process, the video codec may then access (e.g., read) a greater number of samples from the memory than the number of samples in the sub-block. When the size of the block is M×N, accessing M samples in each row from the memory may often involve overhead, such as accessing more than M samples due to a limitation on the minimum number of samples accessed each time. For example, a given memory access process may require the video codec to read samples from a first row before reading samples from another row. In these examples, the video codec may then read a minimum of X samples before accessing a target sample in a narrow sub-block, where X is greater than the number of samples in the narrow sub-block. Due to the narrow shape of 1×N and 2×N sub-blocks, this problem is exacerbated by the large number of rows of narrow sub-blocks. For example, a 4×4 CU may have only four rows. In these examples, even if the sub-block has the same number of samples as a 4×4 CU, the video codec may need to access 16 rows for a 1×16 sub-block. Especially in the worst case where there are several small coded blocks in a high-resolution sequence, the resulting overhead may cause the video codec to experience delays during the coding process. In addition, the reconstruction of the sub-blocks in the CU depends on the reconstruction of other sub-blocks that are preceding in the decoding order in the coded block. In this instance, the video codec may experience increased delays in encoding or decoding the CU due to sub-blocks of this nature.

[0040] The disclosed prediction block grouping technique can solve the above-mentioned problems and other problems by grouping prediction blocks (e.g., sub-blocks) into prediction block groups (PBGs) and reconstructing samples of prediction blocks contained in one PBG independently of samples of other prediction blocks contained in the same PBG. Specifically, intra sub-partitioning (ISP) can be used to divide the coding unit vertically, horizontally, or a combination of horizontal and vertical to form prediction blocks, and then the prediction blocks can be grouped into various PBGs.

[0041] Various techniques in the present disclosure may be described with reference to a video codec or with reference to video coding, which is intended to be a general term that may refer to a video codec and a video encoding or video decoder. Unless otherwise explicitly stated, it should not be assumed that the techniques described with respect to a video encoder or video decoder cannot be performed by the other of the video encoder or video decoder. For example, in many instances, a video decoder performs the same or sometimes opposite coding techniques as a video encoder to decode the encoded video data. In many cases, a video encoder also includes a video decoding loop, and therefore the video encoder performs video decoding as part of encoding the video data. Therefore, unless otherwise stated, the techniques described in the present disclosure with respect to a video decoder may also be performed by a video encoder, and vice versa.

[0042] Figure 1 1 is a block diagram illustrating an example video encoding and decoding system 100 that may perform the techniques of the present disclosure. The techniques of the present disclosure are generally directed to encoding (encoding and / or decoding) video data. Generally speaking, video data includes any data used to process video. Thus, video data may include original unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata (e.g., signaling data).

[0043] like Figure 1 As shown, in this example, the system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. In particular, the source device 102 provides the video data to the destination device 116 via a computer-readable medium 110. The source device 102 and the destination device 116 may include any of a variety of devices, including a desktop computer, a notebook (i.e., a laptop computer), a tablet computer, a set-top box, a handheld phone (such as a smartphone), a television, a camera, a display device, a digital media player, a video game console, a video streaming device, etc. In some cases, the source device 102 and the destination device 116 may be equipped for wireless communication and, therefore, may be referred to as wireless communication devices.

[0044] exist Figure 1In the example of , source device 102 includes video source 104, memory 106, video encoder 200 and output interface 108. Destination device 116 includes input interface 122, video decoder 300, memory 120 and display device 118. According to the present disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 can be configured to apply a technique for generalized intra-frame prediction. Thus, source device 102 represents an example of a video encoding device, and destination device 116 represents an example of a video decoding device. In other examples, source device and destination device can include other components or arrangements. For example, source device 102 can receive video data from an external video source such as an external camera. Similarly, destination device 116 can interface with an external display device instead of including an integrated display device.

[0045] like Figure 1 The system 100 shown is only an example. Generally speaking, any digital video encoding and / or decoding device can perform techniques for generalized intra-frame prediction. The source device 102 and the destination device 116 are only examples of such coding devices, wherein the source device 102 generates coded video data for transmission to the destination device 116. The present disclosure represents a "coding" device as a device that performs data coding (coding and / or decoding). Thus, the video encoder 200 and the video decoder 300 represent examples of coding devices, specifically, a video codec and a video decoder, respectively. In some examples, the devices 102 and 116 can operate in a substantially symmetrical manner so that each of the devices 102 and 116 includes a video coding and decoding component. Therefore, the system 100 can support one-way or two-way video transmission between the video devices 102 and 116, for example, for video streaming, video playback, video broadcasting or video telephony.

[0046] Generally speaking, video source 104 represents a source of video data (i.e., original uncoded video data), and provides a continuous sequence of images (also referred to as "frames") of video data to video encoder 200, which encodes the data of the images. Video source 104 of source device 102 may include a video capture device, such as a camera, a video archive containing previously captured original video, and / or a video feed interface that receives video from a video content provider. As a further alternative, video source 104 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange images from a receiving order (sometimes referred to as a "display order") into a coding order for coding. Video encoder 200 may generate a bitstream containing coded video data. Source device 102 may then output the encoded video data onto computer-readable medium 110 via output interface 108 , to be received and / or retrieved by, for example, input interface 122 of destination device 116 .

[0047] The memory 106 of the source device 102 and the memory 120 of the destination device 116 represent general purpose memories. In some examples, the memories 106, 120 may store raw video data (e.g., raw video from the video source 104) and raw decoded video data from the video decoder 300. Additionally or alternatively, the memories 106, 120 may store software instructions that can be executed by, for example, the video encoder 200 and the video decoder 300, respectively. Although the memory 106 and the memory 120 are shown separately from the video encoder 200 and the video decoder 300 in this example, it should be understood that the video encoder 200 and the video decoder 300 may also include internal memories that achieve functionally similar or equivalent purposes. Further, the memories 106, 120 may store, for example, encoded video data output from the video encoder 200 and input to the video decoder 300. In some examples, portions of the memories 106, 120 may be allocated as one or more video buffers, for example, to store raw, decoded, and / or encoded video data.

[0048] The computer-readable medium 110 may represent any type of medium or device capable of transmitting the encoded video data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to directly send the encoded video data to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, the output interface 108 may modulate a transmission signal containing the encoded video data, and the input interface 122 may demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other equipment that facilitates communication from the source device 102 to the destination device 116.

[0049] In some examples, computer-readable medium 110 may include storage device 112. Source device 102 may output the encoded data from output interface 108 to computer-readable medium 110 (e.g., storage device 112). Similarly, destination device 116 may access the encoded data from computer-readable medium 110 (e.g., storage device 112) via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard disk, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0050] In some examples, computer-readable medium 110 may include file server 114 or another intermediate storage device that can store the encoded video data generated by source device 102. Source device 102 may output the encoded video data to file server 114 or another intermediate storage device that can store the encoded video generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing the encoded video data and sending the encoded video data to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded video data from file server 114 through any standard data connection including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of the two suitable for accessing the encoded video data stored on file server 114. The file server 114 and the input interface 122 may be configured to operate according to a streaming transfer protocol, a download transfer protocol, or a combination thereof.

[0051] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired network components (e.g., Ethernet cards), wireless communication components that operate according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data, such as encoded video data, according to a cellular communication standard such as 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, or the like. In certain examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data, such as encoded video data, according to other wireless standards, such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee TM ), Bluetooth TM Standards, etc., to transmit data such as encoded video data. In some examples, source device 102 and / or destination device 116 may include respective system-on-a-chip (SoC) devices. For example, source device 102 may include an SoC device to perform the functions attributed to video encoder 200 and / or output interface 108, and destination device 116 may include an SoC device to perform the functions attributed to video decoder 300 and / or input interface 122.

[0052] The technology disclosed herein can be applied to video encoding and decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission such as Dynamic Adaptive Streaming over HTTP (DASH), digital video encoded to a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0053] The input interface 122 of the destination device 116 receives the encoded video bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded video bitstream may contain signaling information defined by the video encoder 200 that is also used by the video decoder 300, such as syntax elements with values ​​describing characteristics and / or processing of video blocks or other units of coding (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays a decoded image of the decoded video data to a user. The display device 118 may represent any of a variety of display devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0054] Although not in Figure 1 , but in some examples, each of the video encoder 200 and the video decoder 300 may be integrated with an audio encoder and / or an audio decoder and may include appropriate MUX-DEMUX units or other hardware and / or software to process multiplexed streams containing audio and video in a common data stream. If applicable, the MUX-DEMUX units may conform to the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0055] Each of the video encoder 200 and the video decoder 300 can be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device can store instructions for the software in a suitable non-temporary computer-readable medium and use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Each of the video encoder 200 and the video decoder 300 can be included in one or more encoders or decoders, both of which can be integrated as part of a combined encoder / decoder (CODEC) in each device. The device containing the video encoder 200 and / or the video decoder 300 may include an integrated circuit, a microprocessor, and / or a wireless communication device such as a cellular phone.

[0056] The video encoder 200 and the video decoder 300 may operate according to a video coding standard such as ITU-T H.265, also known as High Efficiency Video Coding (HEVC), or an extension thereof such as a multi-view and / or scalable video coding extension. Alternatively, the video encoder 200 and the video decoder 300 may operate according to other proprietary or industry standards such as the Joint Exploration Model (JEM) or ITU-T H.266, also known as Versatile Video Coding (VVC). The latest draft of the VVC standard is described in: Bross et al., “Versatile Video Coding (Draft 4)”, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 13th Meeting: Marrakech, MA, January 9-18, 2019, JVET-M1001-v2 (hereinafter referred to as “VVC Draft 4”). However, the techniques of the present disclosure are not limited to any particular coding standard.

[0057] Generally speaking, the video encoder 200 and the video decoder 300 can perform block-based coding of an image. The term "block" generally refers to a structure containing data to be processed (e.g., encoded, decoded, or otherwise used in the encoding and / or decoding process). For example, a block may contain a two-dimensional matrix of samples of luminance and / or chrominance data. Generally speaking, the video encoder 200 and the video decoder 300 can code video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, the video encoder 200 and the video decoder 300 can code luminance and chrominance components, where the chrominance components may include red and blue chrominance components, instead of coding red, green and blue (RGB) data of the samples of the image. In some examples, the video encoder 200 converts the received RGB format data into a YUV representation before encoding, and the video decoder 300 converts the YUV representation into an RGB format. Alternatively, a pre-processing and post-processing unit (not shown) can perform these conversions.

[0058] The present disclosure generally refers to coding (e.g., encoding and decoding) of an image, including the process of encoding or decoding image data. Similarly, the present disclosure may refer to coding a block of an image to include the process of encoding or decoding the data of the block, for example, prediction and / or residual coding. The encoded video bitstream generally includes a series of syntax element values ​​used to represent coding decisions (e.g., coding modes) and the partitioning of the image into blocks. Thus, references to coding an image or a block should generally be understood to refer to coding the values ​​of the syntax elements that form the image or block.

[0059] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video codec (such as video encoder 200) partitions a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video codec partitions the CTU and CU into four equal non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without a child node may be referred to as a "leaf node", and the CU of such a leaf node may include one or more PUs and / or one or more TUs. The video codec may further partition the PU and TU. For example, in HEVC, the residual quadtree (RQT) represents the partitioning of a TU. In HEVC, a PU represents inter-frame prediction data, and a TU represents residual data. An intra-predicted CU includes intra-frame prediction information, such as an intra-frame mode indication. The present disclosure may generally use the term partitioning to represent a block as a TU, a PU, or a CU, or to represent multiple blocks or an area containing multiple blocks.

[0060] As another example, the video encoder 200 and the video decoder 300 may be configured to operate according to JEM or VVC. According to JEM or VVC, a video codec (such as the video encoder 200) partitions an image into a plurality of coding tree units (CTUs). The video encoder 200 may partition the CTU according to a tree structure (e.g., a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure eliminates the concept of multiple partition types, such as the distinction between CU, PU, ​​and TU of HEVC. The QTBT structure comprises two levels: a first level partitioned according to quadtree partitioning, and a second level partitioned according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to the coding units (CUs).

[0061] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) partitioning. Ternary tree partitioning is a partitioning that divides a block into three sub-blocks. In some examples, ternary tree partitioning partitions a block into three sub-blocks without partitioning the original block through the center. The partitioning types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0062] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma component and the chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the respective chroma components).

[0063] The video encoder 200 and the video decoder 300 may be configured to use quadtree segmentation, QTBT segmentation, MTT segmentation or other segmentation structures according to HEVC. For illustrative purposes, a description of the technology disclosed herein is provided for QTBT segmentation. However, it should be understood that the technology disclosed herein may also be applied to video codecs configured to use quadtree segmentation or other types of segmentation.

[0064] With respect to sample dimensions in terms of vertical and horizontal dimensions, for example, 16×16 samples or 16 by 16 samples, the present disclosure may use “N×N” and “N by N” interchangeably to represent blocks (such as CUs or other video blocks). Generally speaking, a 16×16 CU will have 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Similarly, an N×N CU generally has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. The samples in a CU may be arranged in rows and columns. Furthermore, a CU does not necessarily have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may include N×M samples, where M is not necessarily equal to N.

[0065] The video encoder 200 encodes the video data of the CU representing prediction and / or residual information and other information. The prediction information indicates how the CU will be predicted in order to form a prediction block for the CU. The residual information generally represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

[0066] In order to predict a CU, the video encoder 200 can generally form a prediction block for the CU by inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting a CU from data of a previously coded image, while intra-frame prediction generally refers to predicting a CU from previously coded data of the same image. In order to perform inter-frame prediction, the video encoder 200 can use one or more motion vectors to generate a prediction block. The video encoder 200 can generally perform a motion search to identify, for example, a reference block that closely matches the CU in terms of the difference between the CU and the reference block. The video encoder 200 can use the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean square difference (MSD), or other such difference calculations to calculate a difference metric to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 can use unidirectional prediction or bidirectional prediction to predict the current CU.

[0067] Some examples of JEM and VVC also provide an affine motion compensation mode, which can be considered an inter-prediction mode. In the affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion, such as enlargement or reduction, rotation, perspective motion, or other irregular motion types.

[0068] To perform intra prediction, the video encoder 200 may select an intra prediction mode to generate a prediction block. Some examples of JEM and VVC provide sixty-seven intra prediction modes, including modes for various directions as well as a planar mode and a DC mode. Generally speaking, the video encoder 200 selects an intra prediction mode that describes neighboring samples of a current block (e.g., a block of a CU) from which to predict the prediction samples of the current block. Assuming that the video encoder 200 encodes CTUs and CUs in a raster scan order (from left to right, from top to bottom), such samples may generally be above, to the upper left, or to the left of the current block in the same image as the current block.

[0069] The video encoder 200 encodes data representing the prediction mode of the current block. For example, for inter-frame prediction mode, the video encoder 200 can encode data indicating which of the various available inter-frame prediction modes is used and the motion information of the corresponding mode. For unidirectional or bidirectional inter-frame prediction, for example, the video encoder 200 can encode the motion vector using advanced motion vector prediction (AMVP) or merge mode. The video encoder 200 can use a similar mode to encode the motion vector of the affine motion compensation mode.

[0070] After prediction (such as intra-frame prediction or inter-frame prediction of a block), the video encoder 200 can calculate residual data for the block. The residual data (such as a residual block) represents the sample-by-sample difference between the block and the prediction block of the block, and the prediction block is formed using the corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to generate transform data in the transform domain rather than the sample domain. For example, the video encoder 200 can apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 can apply a secondary transform after a primary transform, such as a mode-dependent non-separable secondary transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), etc. The video encoder 200 generates transform coefficients after applying one or more transforms.

[0071] As described above, after performing any transforms to produce transform coefficients, the video encoder 200 may perform quantization on the transform coefficients. Quantization generally refers to the process of quantizing the transform coefficients to possibly reduce the amount of data used to represent the transform coefficients, thereby providing further compression. By performing the quantization process, the video encoder 200 may reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 may round down an n-bit value to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 may perform a bitwise right shift of the value to be quantized.

[0072] After quantization, the video encoder 200 can scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix containing the quantized transform coefficients. The scan can be designed to place the transform coefficients with higher energy (and therefore lower frequency) in front of the vector and the transform coefficients with lower energy (and therefore higher frequency) in the back of the vector. In some examples, the video encoder 200 can scan the quantized transform coefficients using a predefined scan order to generate a serialized vector, and then entropy encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 can perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 can, for example, entropy encode the one-dimensional vector according to context adaptive binary arithmetic coding (CABAC). The video encoder 200 can also entropy encode the value for the syntax element, which describes the metadata associated with the encoded video data used by the video decoder 300 in decoding the video data.

[0073] To perform CABAC, the video encoder 200 may assign context within a context model to a symbol to be transmitted. For example, the context may relate to whether the neighboring values ​​of the symbol are zero values. The probability determination may be based on the context assigned to the symbol.

[0074] The video encoder 200 may further generate syntax data, such as block-based syntax data, image-based syntax data, and sequence-based syntax data, to the video decoder 300, for example, in a picture header, a block header, a slice header, or generate other syntax data, such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS). The video decoder 300 may similarly decode such syntax data to determine how to decode the corresponding video data.

[0075] In this way, the video encoder 200 can generate a bitstream containing encoded video data, for example, syntax elements describing the partitioning of an image into blocks (eg, CUs) and prediction and / or residual information for the blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0076] In general, the video decoder 300 performs the reverse process performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 may decode the values ​​of the syntax elements of the bitstream using CABAC in a manner substantially similar to (although reversed from) the CABAC encoding process of the video encoder 200. The syntax elements may define partitioning information for partitioning an image into CTUs and partitioning each CTU according to a corresponding partitioning structure, such as a QTBT structure, to define CUs of the CTU. The syntax elements may further define prediction and residual information for a block (e.g., CU) of video data.

[0077] For example, the residual information may be represented by quantized transform coefficients. The video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of the block to reproduce a residual block for the block. The video decoder 300 uses the signaled prediction mode (intra-frame or inter-frame prediction) and related prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for the block. The video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the original block. The video decoder 300 may perform additional processes (such as performing a deblocking process) to reduce visual artifacts along block boundaries.

[0078] According to the technology of the present disclosure, the video encoder 200 and / or the video decoder 300 may perform generalized intra-frame prediction. For example, the video encoder 200 and / or the video decoder 300 may encode an indication of the performance of generalized prediction of a current coded block of video data; divide the current coded block into one or more prediction blocks based on the indication; define one or more prediction block groups (PBGs) based on the one or more prediction blocks; determine the coding order of the PBGs; and code the PBGs in the coding order.

[0079] In some examples, a video codec (e.g., video encoder 200 and / or video decoder 300) may use intra-frame sub-partitioning (ISP) to perform partitioning of a coding unit (CU) of video data to form a set of prediction blocks. For example, the video codec may perform vertical partitioning or horizontal partitioning of the CU. In another example, the video codec may perform the partitioning of the CU as a combination of vertical and horizontal partitioning of the CU. For example, the video codec may perform vertical partitioning on a portion of the CU and horizontal partitioning on other portions of the CU. In any case, the video codec may partition the CU via various partitioning or segmentation techniques to form a set of prediction blocks.

[0080] In some examples, the prediction block may include at least a first prediction block and a second prediction block. In some examples, a CU may include a single prediction block. The prediction block may include a narrow vertical block, a narrow horizontal block, or a combination of a vertical block and a horizontal block. It should be noted that the term "narrow" herein generally includes non-square blocks (e.g., rectangular blocks, etc.).

[0081] In addition, the video codec may group multiple prediction block sets into a first prediction block group (PBG). In some examples, the PBG may include at least two prediction blocks. For example, the PBG may include one or more vertical prediction blocks, one or more horizontal prediction blocks, or one or more combinations of vertical prediction blocks and horizontal prediction blocks. In another example, the PBG may include a single prediction block. In addition, the CU may include PBGs of different sizes. For example, a single CU may include multiple PBGs, wherein the first PBG is formed by grouping one or more prediction blocks (e.g., vertical, horizontal, or both vertical and horizontal blocks), and the second PBG is formed by grouping at least two prediction blocks (e.g., vertical, horizontal, or both vertical and horizontal blocks). In some examples, a single CU may further include a third PBG formed by grouping at least two prediction blocks (e.g., vertical blocks, horizontal blocks, or both vertical and horizontal blocks), a fourth PBG formed by grouping at least three prediction blocks (e.g., vertical, horizontal, or both), and the like.

[0082] In addition, one or more subsequent CUs may include the same or different arrangements of prediction blocks and / or PBGs. For example, a second CU may include a partitioning that is the same or at least similar to the partitioning of one or more previous CUs. In another example, a second CU may include a partitioning scheme that is different from the partitioning scheme of one or more previous CUs. For example, a video codec may partition a first CU using vertical partitioning, partition a second CU using horizontal partitioning, partition a third CU using a combination of horizontal and vertical partitioning, partition a fourth CU using different types of horizontal partitioning (e.g., to form a more or less narrow prediction block relative to other partitioned prediction blocks), or any other combination or variation.

[0083] Although the present invention describes certain examples in terms of vertical partitioning or, in some cases, horizontal partitioning, it will be understood that the techniques of the present invention are not limited to as described. In an additional non-limiting example and for illustrative purposes, a video decoder may partition multiple CUs using the same partitioning scheme, such as by partitioning at least two CUs using vertical partitioning. The video codec may additionally group the partitioned prediction blocks in a different or the same manner for each partitioned CU. For example, the video codec may perform vertical partitioning on the first CU to form a group of at least three PBGs, while the video codec may perform similar vertical partitioning on one or more subsequent CUs, but form at least two PBGs. That is, from one partitioned CU to the next partitioned CU, the number of PBGs of each CU may not necessarily be equal, even if the video codec uses, for example, vertical partitioning to partition each CU. It will be understood that for the sake of brevity, not all options may be explicitly depicted, but other partitioning and / or grouping options may be implemented by one or more of the various techniques or variations of the techniques described in the present disclosure. In addition, the video codec may perform independent reconstruction of such PBGs, as described above and further described below.

[0084] The video codec may then reconstruct the samples of the prediction block included in the first PBG independently of the samples of other prediction blocks included in the first PBG. For example, the video codec may reconstruct the samples as part of a decoding loop. In other cases, the video decoder 300 may partition the CU to form a set of prediction blocks and group the prediction blocks into PBGs.

[0085] In general, the present disclosure may refer to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to the communication of values ​​for syntax elements and / or other data used to decode encoded video data. That is, the video encoder 200 may signal the values ​​of syntax elements in the bitstream. In general, signaling refers to generating values ​​in the bitstream. As described above, the source device 102 may transmit the bitstream to the destination device 116 in substantially real time (or non-real time, such as may occur when storing syntax elements to the storage device 112 for later retrieval by the destination device 116).

[0086] Figure 2is a diagram illustrating exemplary intra prediction modes. An example of intra prediction is wide-angle intra prediction. In some examples, the intra prediction of a luminance block includes 35 modes, including a planar prediction mode, a DC prediction mode, and 33 angular (or directional) prediction modes. For example, in some cases, the directional prediction for a square block can use directions between -135 degrees and 45 degrees of the current block 250, such as in the Versatile Video Coding Test Model 2 (VTM2) (see "Algorithm Description of Versatile Video Coding and Test Model 2 (VTM2)" by J. Chen, Y. Ye, and S. Kim et al., 11th JVET Meeting, Ljubljana, SI, July 2018, JVET-K1002), as Figure 2 shown. The 35 modes of intra prediction are indexed as shown in Table 1 below.

[0087] Table 1 - Specification of Intra Prediction Modes and Associated Names

[0088]

[0089]

[0090] In VTM2, the block structure for specifying the prediction block for intra prediction is not limited to a square (width w = height h). For example, a rectangular prediction block (w > h or w < h) can increase the coding efficiency based on the characteristics of the content.

[0091] In such a rectangular block, restricting the direction of intra prediction to the range of -135 degrees to 45 degrees can result in the situation of using farther reference samples instead of closer reference samples for intra prediction. This design may have an impact on the coding efficiency. Relaxing the restricted range may be more beneficial, such that closer reference samples (beyond -135 to 45 degrees) can be used for prediction. An example of this situation is given in Figure 3 below.

[0092] Figure 3 is a block diagram illustrating an example of an 8×4 rectangular block. In Figure 3 the example, because the direction of intra prediction is restricted to the range of -135 degrees to 45 degrees, the "closer" reference samples (indicated by circles 3002) are not used, but the "farther" reference samples (indicated by circles 3006) can be used. That is, in Figure 3 the example, some reference samples within -135 degrees but farther than those indicated by circles 3002 can be used, instead of some reference samples that are closer than other samples, for example, closer than the samples indicated by circles 3006.

[0093] Figures 4A-4Cis a block diagram providing an illustration of a pattern mapping process for patterns outside of a diagonal direction range. Figure 4A In the example of FIG. 4 , the square block CU 402 does not require angle mode remapping. Figure 4B Angular mode remapping for a horizontal non-square block CU 404 is illustrated. Figure 4C Angular mode remapping for a vertical non-square block CU 406 is illustrated.

[0094] Figure 5 is a block diagram illustrating wide-angle intra prediction employed in VTM2. In some examples, wide-angle intra prediction includes angular intra prediction for square and non-square blocks, such as block 550.

[0095] During the 12th JVET meeting, modifications to wide-angle intra prediction were adopted into VVC Test Model 3 (VTM3), details of which are available from: (i) L. Zhao, X. Zhao, S. Liu, and X. Li, “CE3-related: Unification of angular intra prediction for square and non-square blocks,” 12th JVET meeting, Macau SAR, CN, October 2018, JVET-L0279; (ii) J. Chen, Y. Ye, and S. Kim, “Algorithmic description of the generic video codec and test model 3 (VTM3),” 12th JVET meeting, Macau SAR, CN, October 2018, JVET-L1002; and (iii) B. Bross, J. Chen, and S. Liu, “Generic video codec (draft 3),” 12th JVET meeting, Macau SAR, CN, October 2018, JVET-L1002. SAR, CN, October 2018, JVET-L1001.

[0096] The proposed approach includes two modifications to unify the angular intra prediction for square and non-square blocks. First, the angular prediction direction can be modified to cover the diagonal direction for all block shapes. Second, for all block aspect ratios (square and non-square), the angular direction can be kept in the range between the lower left diagonal direction and the upper right diagonal direction, e.g. Figures 4A-4C In addition, for all block shapes, the number of reference samples in the top reference row and left reference column is limited to 2*width+1 and 2*height+1. Figure 5 In the example of , wide angles (-1 to -10, and 67 to 76) are depicted in addition to the 65 angle mode.

[0097] Figure 6650 is a block diagram illustrating a wider angle of prediction for a block 650 employed in VTM3. Although VTM3 defines 95 modes for any block size, only 67 modes are allowed. The exact modes allowed depend on the ratio of the block width to the block height. This is achieved by limiting the range of modes for a specific block size. Figure 6 A plot of the wide angles (-1 to -14, and 67 to 80) in VTM3 beyond modes 2 and 66 for a total of 93 angular modes is provided.

[0098] Figure 7 is an exemplary mapping table providing an exemplary mapping between predModeIntra and the angle parameter intraPredAnglein VTM3, further details of which can be obtained from: B. Bross, J. Chen and S. Liu, "Generic Video Codec (Draft 3)", 12th JVET Meeting, Macau SAR, CN, October 2018, JVET-L1001. In the following, angle modes with positive intraPredAngle values ​​are referred to as positive angle modes (mode index <18 or >50), while angle modes with negative intraPredAngle values ​​are referred to as negative angle modes (mode index >18 and <50).

[0099] As shown in the following equation (1), the inverse angle parameter invAngle is derived based on IntraPredAngle:

[0100]

[0101] Note that intraPredAngle values ​​that are multiples of 32 (ie, values ​​of 0, 32, 64, 128, 256, 512 in this example) correspond to predictions from non-fractional reference array samples, as is the case in the VTM3 specification. Table 2 below illustrates the diagonal patterns corresponding to various block aspect ratios.

[0102] Table 2 - Diagonal patterns corresponding to various block aspect ratios.

[0103]

[0104]

[0105] Intra-frame sub-partition coding is described in the following sections of the present disclosure. Intra-frame sub-partition coding (ISP) (S. DeLuxán Hernández, H. Schwarz, D. Marpe, T. Wiegand (HHI) "CE3: Row-based intra-frame coding mode", JVET-L0076) is a method of dividing or partitioning a coding block (e.g., a coding unit) into multiple sub-blocks. For example, a coding unit can be divided or partitioned into two sub-blocks, four sub-blocks, etc. In some instances, the sub-blocks may be referred to as prediction blocks. In some examples, each sub-block within the coding block is reconstructed in decoding order before reconstructing subsequent sub-blocks in decoding order. In the fourth working draft of VVC (hereinafter referred to as "VVC WD4" and may be in phenix.it-sudparis.eu / jvet / doc_end_user / documents / 13_Marrakech / wg11 / JVET-M1001-v2.zip In such an example, the reference samples for these ISP coded blocks are restricted to the reference row from the closest coded block (refer to MRLIdx=0, as discussed below with reference to multiple reference row prediction).

[0106] Figure 8 and 9 is a block diagram illustrating an exemplary partitioning of blocks for ISP coding. Figure 8 Examples of partitioning of 4x8 and 8x4 blocks are illustrated. Fig. 9 An example of partitioning of all blocks except 4×8, 8×4, and 4×4 blocks is illustrated.

[0107] One bit is used to signal whether the coded block is divided into intra sub-divisions, and the second bit is used to indicate the division type. Exemplary division types include horizontal division, vertical division, vertical and horizontal division, etc. Based on the intra mode and division type used, two different types of processing orders can be used, which are called normal order and reverse processing order. In the normal order, the first sub-division to be processed is the sub-division containing the upper left sample point of the CU, and then continues to the sub-division downward (horizontal division) or to the right (vertical division). On the other hand, the reverse processing order either starts with the sub-division containing the lower left sample point of the CU and continues upward (horizontal division), or starts with the sub-division containing the upper right sample point of the CU and continues to the left (vertical division).

[0108] In some examples, a variation of the ISP may use only the normal processing order, such as in VVC WD4. It should be noted that the terms "sub-block" and "sub-partition" are used interchangeably herein. In any case, the two terms refer to the blocks obtained by partitioning a coded block using the ISP.

[0109] Some syntax (eg, Table 3 and Table 4) and semantics associated with ISP in VVC WD4 are shown below in italics to indicate the relevant syntax:

[0110] Table 3 - Syntax table of coding unit

[0111]

[0112] Table 4 - Syntax table of transformation tree

[0113]

[0114]

[0115]

[0116] Semantics of coding units

[0117] intra_subpartitions_mode_flag[x0][y0] equal to 1 specifies that the current intra coding unit is partitioned into NumIntraSubPartitions[x0][y0] rectangular transform block subpartitions. intra_subpartitions_mode_flag[x0][y0] equal to 0 specifies that the current intra coding unit is not partitioned into rectangular transform block subpartitions.

[0118] When intra_subpartitions_mode_flag[x0][y0] is not present, the value of intra_subpartitions_mode_flag is inferred to be equal to 0.

[0119] In some examples, the intra-subpartition split flag may be used to indicate a split type or multiple split types. For example, intra_subpartitions_split_flag[x0][y0] may specify a split type or the number of various split types. In some examples, the intra-subpartition split flag may indicate whether the intra-subpartition split type is horizontal, vertical, rectangular, a mixture of vertical and horizontal, or any other split type. When intra_subpartitions_mode_flag[x 0][y 0] is not present, the video codec may infer that the flag is equal to 0.

[0120] The variable IntraSubPartitionsSplitType specifies the partition type for the current luma coding block, as shown in Table 5 below. IntraSubPartitionsSplitType is derived as follows:

[0121] If intra_subpartitions_mode_flag[x0][y0] is equal to 0, IntraSubPartitionsSplitType is set equal to 0.

[0122] Otherwise, IntraSubPartitionsSplitType is set equal to 1+intra_subpartitions_split_flag[x0][y0].

[0123] Table 5 - IntraSubPartitionsSplitType name association

[0124] IntraSubPartitionsSplitType Name of IntraSubPartitionsSplitType 0 ISP_NO_SPLIT 1 ISP_HOR_SPLIT 2 ISP_VER_SPLIT

[0125] The variable NumIntraSubPartitions specifies the number of transform block subpartitions that the intra luma codec block is partitioned into. NumIntraSubPartitions is derived as follows:

[0126] - If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT, set NumIntraSubPartitions equal to 1.

[0127] Otherwise, if either of the following conditions is true, then NumIntraSubPartitions is set equal to 2: (1) cbWidth is equal to 4 and cbHeight is equal to 8, or (2) cbWidth is equal to 8 and cbHeight is equal to 4. Otherwise, NumIntraSubPartitions is set equal to 4.

[0128] Multiple reference row prediction will be described in the following sections of this disclosure. Samples within the neighborhood of a coded block are used for intra prediction of the block. Typically, the reconstructed reference sample rows closest to the left and top boundaries of the coded block are used as reference samples for intra prediction. In some examples, for example, in VVC WD4, the video codec may use other samples as reference samples, such as samples in the neighborhood of a coded block or CU.

[0129] Fig.10 is a block diagram illustrating reference sample rows that can be used for intra prediction. For each coded block, an index indicating the reference row used is signaled.

[0130] In some examples, such as in VVC WD4, the video codec may only use reference rows with MRLIdx equal to 0, 1, or 3. The video codec may encode with a truncated unary codeword the index to the reference row used to encode the block (e.g., values ​​0, 1, and 2 indicating rows with MRLIdx 0, 1, and 3, respectively). Planar mode and DC mode are not used for reference rows with MRLIdx>0 that are used.

[0131] In some examples, intra sub-partitioning (ISP) coding of a block (e.g., coding unit (CU)) may result in sub-blocks (e.g., prediction blocks) having very small sizes in one or more dimensions. For example, when a video codec encodes a coding block of size 4×N (W×H) using ISP coding and uses vertical partitioning, the size of the resulting sub-block is 1×N when N is greater than 8. In implementations that rely on row-wise storage of samples, increasingly narrower sub-blocks may result in memory accesses of more samples than the number of samples in the sub-block. When the size of the block is M×N, accessing M samples in each row from memory may often involve overhead, such as accessing more than M samples due to a limit on the minimum number of samples per access.

[0132] Due to the narrow shape of 1×N and 2×N blocks, this problem may be exacerbated by the large number of rows of narrow blocks. For example, a 4×4 block may have only four rows. Although a 1×16 sub-block has the same number of samples as a 4×4 block, there are 16 rows to be accessed. In delay critical cases, especially in the worst case where there are several small coded blocks in a high resolution sequence, the resulting overhead may be problematic. Because the reconstruction of a sub-block in a coded block may depend on the reconstruction of other sub-blocks in coded blocks preceding in the decoding order, these sub-blocks may increase the delay for decoding the block, which may be undesirable.

[0133] Some solutions to the problem have been proposed. However, each of the proposed solutions involves drawbacks that may lead to inefficient coding. One proposed solution is to disable vertical partitioning of 4×N and 8×N blocks, which results in blocks of size 1×N and 2×N, respectively. However, the proposed solution may reduce the efficiency of the ISP. Another proposed solution (i.e., in S.De V.George, J.Ma, T.Nguyen, H.Schwarz, D.Marpe, T.Wiegand (HHI), "CE3: Intra-sub-partition coding mode", JVET-M0102) is to restrict the prediction of 1×N sub-blocks to the VER prediction mode so that they do not depend on the reconstruction of other sub-blocks in the coding block. However, the proposed solution also leads to restrictions on the intra-frame modes allowed for the coding block, which leads to low coding efficiency.

[0134] This disclosure proposes techniques for improving the design of sub-partition coding of a CU.One or more of the techniques disclosed herein may be implemented alone or together in any combination.

[0135] According to the proposed first technique, a video codec (e.g., video encoder 200 and / or video decoder 300) may perform generalized prediction of a sub-block (e.g., a prediction block). As an example, the video codec may encode an indication (e.g., a syntax element) to perform generalized prediction on a coded block. In some examples, the indication to perform generalized prediction may involve an indication to encode the coded block with an ISP. In some examples, the indication to perform generalized prediction may be applied only to blocks with specific characteristics, for example, when the block width is within a threshold range, or when the block height is within a threshold range, or when the number of samples in the block is within a threshold range, or when the sum of the block width and the block height is within a threshold range, or when the minimum or maximum value of the block width and the block height is within a threshold range. In some examples, for a block, a subset of intra modes may be defined from a set of allowed intra modes for the block. In these examples, generalized prediction may be applied to the block only when the block is encoded with an intra mode belonging to the subset. In some examples, the indication may be inferred by block properties / characteristics, intra-modes, and properties of neighboring blocks, or may not be explicitly signaled. For example, when the block width is less than a certain threshold, generalized prediction may be applied.

[0136] As another example, based on an indication to perform generalized prediction, the video codec may partition the coding unit into one or more prediction blocks.

[0137] As another example, a video codec may define one or more prediction block groups (PBGs). A PBG may contain one or more prediction blocks. The prediction of any sample in a PBG may not directly depend on the value of the prediction of any other sample in the PBG. For example, a video codec may reconstruct samples of a prediction block contained in a first PBG independently of samples of other prediction blocks contained in the same PBG. In this way, the prediction of samples in a first PBG does not depend on the values ​​of the predictions of other samples in the first PBG.

[0138] In some examples, each PBG contains exactly one prediction block. In some examples, each PBG may contain n PB In these examples, the video codec may determine n based on the width and height of the coding unit. PB In another example, the video codec may determine n based on the number of samples of the coding unit or based on the number of samples and the width and height of the coding unit. PBIn some examples, the video codec may also use the partition type (e.g., horizontal, vertical, rectangular, a mixture of horizontal and vertical, etc.) and / or the intra mode (e.g., angle, etc.) to determine the n of the coding unit or coding block. PB In some examples, the video codec may specify a prediction block that is not rectangular in shape.

[0139] In some examples, the restriction that "the prediction of any sample in the PBG may not depend on the predicted value of any other sample in the PBG" may be relaxed. For example, when certain filtering operations are applied to a prediction block that may result in correlation between the prediction values ​​of samples within the PBG, the restriction on the prediction correlation of samples within the PBG may still be considered valid as long as conventional intra prediction (without post-prediction filtering operations) does not result in correlation. For example, the restriction may not apply to correlation due to position dependent intra prediction combination (PDPC), boundary filtering, or other similar operations. In some examples, each prediction block in a coded block may be contained in one and only one PBG. In some examples, the restriction may be stricter. For example, it may be restricted that each prediction block in a coded block must be contained in one and only one PBG.

[0140] As another example, a video codec may specify a decoding order for PBGs; the decoding order may be defined such that when PBG P1 precedes PBG P2 in the decoding order, samples in P1 do not depend on samples in P2.

[0141] As another example, for each PBG, the video codec may specify a set of available reference samples (described below).

[0142] As another example, the video codec may obtain prediction values ​​for samples in the PBG based on available reference samples of the PBG and the intra mode used for prediction.

[0143] According to the proposed second technique, the video codec may perform generalized planar prediction. As an example, the video codec may define a set of prediction blocks for a coded block. In some examples, the video codec may code one or more of the prediction blocks in a planar mode. The video codec may perform a determination to divide the coded block into planar prediction blocks by one or more of the following:

[0144] 1. An indication in the bitstream (e.g., a flag) indicating that the block is to be divided into plane prediction blocks. In some examples, the coded block may be indicated to be coded via plane mode. In another example, the indication may be to code the coded block using ISP.

[0145] 2. Planar prediction blocks can be represented by separate mode values ​​M PPB To indicate. When indicating, use M PPB When encoding a block, the block is divided into plane prediction blocks.

[0146] 3. The determination may be inferred based on certain block characteristics and other syntax elements in the bitstream. For example, a block with a certain maximum number of samples and coded with ISP may be partitioned into planar prediction blocks.

[0147] In some examples, the video codec may determine the number N of planar prediction blocks by one or more of the following: PPB And the division decision: N PPB It can be signaled in the bitstream (e.g., for all blocks or for blocks with specific characteristics) or derived based on block characteristics and other syntax elements in the bitstream. PPB Can be set equal to the number of sub-partition blocks within the frame of the coded block. In other examples, N PPB The value of may be set equal to 4. The partitioning direction or method may be determined by block characteristics or derived from syntax elements in the bitstream.

[0148] In some examples, a block may be divided horizontally into N PPB The decision of horizontal partitioning can be determined, for example, by whether the horizontal partitioning direction is indicated.

[0149] Fig.11 is a conceptual diagram illustrating an example of how a coding block may be divided into four horizontal plane prediction blocks.

[0150] In some examples, the video codec may divide the block vertically into N PPB For example, the video codec may determine whether to vertically split based on whether the vertical split direction is indicated.

[0151] Fig.12 is a conceptual diagram illustrating an example of how a coding block may be divided into four vertical plane prediction blocks.

[0152] In some examples, the video codec may partition the block horizontally and vertically, resulting in a total of N PPB A plane prediction block. The video codec may determine whether to split horizontally and vertically based on one or more characteristics of the block (e.g., characteristics of the CU). In some examples, when the split direction is not indicated, the video codec may determine whether to split horizontally and vertically.

[0153] In some examples, the block can be divided into N PPBrectangular regions that share the same upper left sample point as the coded block, and each rectangular region contains the rectangular region before it continuously. Starting from the smallest block defined as the first plane prediction block, the samples belonging to the next rectangular region but not to the previous rectangular region are said to constitute the subsequent plane prediction block. Fig.13 is a conceptual diagram illustrating an example of such a planar prediction block structure.

[0154] In some examples, the video codec may also predict the planar prediction block by a method similar to the planar prediction mode.

[0155] In some examples, the video codec may define a coding order for the prediction blocks. For example, the coding order may be defined by considering the following (a. to d.):

[0156] a. Assign a representative sample to each prediction block; for example, the top left sample of the prediction block. In some cases (non-rectangular blocks), the top left sample may not be defined, and the representative sample may be designated as the leftmost sample in the top row of samples of the prediction block. It must be understood that other definitions of representative samples may also be used to specify the coding order.

[0157] b. Define the scanning order of representative samples (eg, raster scan, zigzag scan).

[0158] c. A prediction block with a representative sample earlier in the scanning order is coded earlier than a prediction block with a representative sample later in the scanning order.

[0159] d. A similar definition of the coding order can also be specified for the prediction block group.

[0160] In some examples, the video codec may specify a plane prediction block group (plane PBG) that includes one or more plane prediction blocks, similar to the concept of the PBG discussed above.

[0161] The boundaries between planar prediction blocks can be smoothed by applying filtering across the prediction block boundaries within the coding block. An example of such filtering is low pass filtering. A second example is the process performed in PDPC mode. A third example is a deblocking filter process. These processes can be applied as a second process step after prediction or as a post-processing step.

[0162] It should be understood that the above examples of plane prediction blocks are only some examples for illustrating the concept. The samples in the coded block can be divided into spatial regions defined as plane prediction blocks, and one or more methods disclosed in the present application can be applied to these spatial regions.

[0163] Furthermore, the term block (coding block, prediction block, etc.) as used herein may apply to luma blocks or chroma blocks (or both), or generally to coding a block of any component of a video.

[0164] According to the third proposed technique, the video codec can determine the availability of reference samples. For each PBG, the available reference samples can be specified as follows:

[0165] In some examples, the set of available reference samples is the same for all prediction blocks in a coded block. Such a system may be implemented in a variety of ways. As an example, the reference sample derivation process remains unchanged and changes are made to the prediction process. The set of available reference samples is derived once for a coded block. In some examples, changes may be made to the prediction process to ensure that the reference samples may not be immediately adjacent to the prediction block, or as specified by the MRLIdx relative to the prediction block.

[0166] In some examples, the reference sample derivation is changed and the prediction process is not changed. In such examples, the available reference samples for the prediction block are derived from the set of available reference samples for the coded block. This derivation may result in derived values ​​for the reference samples of the prediction block at positions corresponding to the reference sample rows used (i.e., based on the value of MRLIdx).

[0167] In some examples, prediction samples from previous PBGs in decoding order are marked as unavailable.

[0168] In some examples, in addition to reference samples from neighboring coded blocks, processed values ​​of a previously coded PBG of the same coded block may be used for intra prediction of the PBG. As an example, the processed values ​​refer to reconstructed samples of the PBG. As another example, the processed values ​​refer to reconstructed samples of the PBG before loop filtering / post filtering operations. As another example, the processed values ​​refer to predicted samples of the PBG.

[0169] According to the proposed fourth technique, the video codec can perform transform coding. The prediction of the samples in the block can be performed by one or more methods described herein. However, the techniques in this section can also be applied to coding blocks that are not predicted using generalized prediction as described above.

[0170] In some examples, the video codec may apply a transform to the residual samples in the PBG. A set of applicable shapes may be defined so that the video codec may apply the transform to all samples of the applicable shapes. For example, the applicable shapes may include square and / or rectangular shapes. In some examples, transform coding of the block may be performed by treating the entire coded block as a block. In these examples, only one transform coding may be applied to the samples in the coded block, regardless of the number of prediction blocks defined therein. In some examples, when the shape of the PBG is an applicable shape on which a transform may be applied, the video codec may apply the transform coding to the PBG. For example, if the samples in the PBG form a rectangular area, the transform may be applied to the samples of the PBG. In some examples, the video codec may apply the transform coding to one or more subsets of the samples in the PBG, so that the samples in each such subset form an applicable shape. As an example, the subsets of the PBG may be constrained to be mutually exclusive. As another example, the subsets of the PBG may be collectively constrained to contain all the samples of the PBG.

[0171] In some examples, applying a transform may also include a transform skip decision for a block, whereby sample values ​​of the block are unchanged, an identity transform is applied, or a scaled identity transform is applied. As an example, a video codec may encode some prediction blocks in a PBG using transform skip. Additionally, a video codec may encode some prediction blocks with other types of transforms. The decision to apply a transform kernel or a transform skip may be inferred (e.g., based on block characteristics, intra-frame modes, neighboring blocks) or signaled in the bitstream.

[0172] In some examples, a sub-block partitioning scheme different from that of the specified prediction block may be defined for transform coding. A transform block group (TBG) may be defined using transform sub-blocks similar to those defined for the prediction block. One or more of the above techniques may be applied to a TBG. For example, when the intra-frame sub-partitioning results in sub-blocks of size 2×N (the size of the prediction block may be 2×N), the transform block may be defined as 1×N, and each transform block may be independently transform coded. In another example, the size of the prediction block may be 2×N, the size of the prediction block group may be 4×N, and the size of the transform block may be 4×N. In another example, for an 8×N coding block, the size of the prediction block may be 2×N, the size of the prediction block group may be 4×N, and the size of the transform block may be 2×N.

[0173] According to the proposed fifth technique, the video codec may not allow certain intra modes. Several intra modes (e.g., angle and non-angle) are defined for intra prediction of blocks. For PBG, only a subset of intra modes may be specified as allowed modes. In other words, certain intra modes may not be allowed for PBG. The determination of the disallowed intra modes may be achieved by considering one or more of the following: (1) the partitioning direction, (2) the coding block width, height, or a function of the width and height, (3) the prediction block width, height, or a function of the width and height, or (4) the above-mentioned block characteristics of adjacent coding blocks, and the intra modes used to encode adjacent coding blocks.

[0174] It should be understood that the above description (including examples) is in no way limiting of the techniques involved and can be combined with other methods not described in this document. One or more determinations or indications may also depend on other conditions, including tile group type, partition type, characteristics of one or more neighboring blocks (width, height, intra mode, prediction type), etc.

[0175] Following are two specific examples of the above techniques.

[0176] In the first example, a prediction block group (PBG) is specified to have a minimum width of four samples. For example, a vertical partition of a 4×N coded block may result in sub-blocks of width 1 or 2, while a vertical partition of an 8×N coded block may result in sub-blocks of width 2 or 4.

[0177] Fig.14 1 is a block diagram illustrating how a coding block 1402 is divided into four prediction blocks (PB1-PB4). In an illustrative example, the coding block 1402 may be an 8×N coding block that a video codec uses an ISP and is coded using vertical partitioning. In this example, the size of prediction blocks PB1, PB2, PB3, and PB4 is 2×N. Prediction block groups PBG1 and PBG2 are defined as having a minimum width (nW) of 4 samples. For example, PBG1 is a prediction block group of size 4×N, and PBG2 is also a second prediction block group of size 4×N. Samples of PBG1 are predicted based on adjacent reference samples. In some examples, the predicted values ​​of samples in PBG1 may not depend on other sample values ​​in PBG1. Samples of PBG2 are predicted based on adjacent reference samples and based on predicted samples of PBG1.

[0178] In some examples, a transform block (eg, a transform unit (TU)) may be defined to have the same size as a prediction block. For example, a video codec may determine the size of a TU to be equal to the size of any one of the prediction blocks included in a PBG.

[0179] In some examples, samples of PBG2 may be predicted based on reference samples from neighboring coding blocks and based on reconstructed samples belonging to the region of PBG1, or based only on reference samples from neighboring coding blocks.

[0180] Fig.15 1 is a block diagram illustrating an example in which a video codec may code a code block 1502 using ISP and vertical partitioning. In an illustrative example, the code block 1502 may be a 4×N code block. In such an example, the size of the prediction block (PB1-PB4) may be 1×N. In some examples, a PBG may be defined as having a width of at least four samples. In such an example, the video codec may group the four prediction blocks (PB1-PB4) into a prediction block group (e.g., PBG1). Similarly, PBG1 may be a PBG of size 4×N.

[0181] In some cases, generalized prediction (designating prediction blocks and prediction block groups as described herein) is applied only to a subset of blocks that meet one or more criteria, which may depend on block size, block shape, subblock width and height, intra mode, etc. For example, the condition may be applied only to coded blocks that generate subblocks with a width less than or equal to 2. In other cases, the condition may be applied to subblocks with a total number of samples less than or equal to 32 samples.

[0182] In another example, samples of all prediction blocks of a coding block are predicted based on reconstructed reference samples of neighboring coding blocks. In these examples, there is no dependency on prediction within the prediction blocks of the coding unit. In some examples, the transform unit size may be equal to the size of the prediction block.

[0183] Fig.16 1 is a conceptual diagram illustrating an example of dividing an M×N coding block 1602 into four prediction blocks (e.g., PB1-PB4). In the illustrative example, the prediction blocks (PB1-PB4) have a size of (M / 4)×N. Reference samples 1604 are used as reference samples for the four prediction blocks. There is no correlation between the prediction samples of one prediction block and the prediction samples of another prediction block. For example, there is no correlation between the prediction samples of PB1 and the prediction samples of PB2, PB3, or PB4.

[0184] In some cases, the four predicted sub-blocks may be predicted in parallel. In another example, the four sub-blocks (PB1-PB4) may be considered together as one block for prediction. For example, the four sub-blocks (PB1-PB4) may be considered together as one PBG for prediction.

[0185] In some examples, the video codec may predict samples of some coding blocks using at least one prediction block group having a width of four samples and predict some other coding units using restricted intra modes. The following restrictions are added to the specification for coding 4×N coding blocks and 8×N coding blocks coded with ISP.

[0186] For a 4×N coded block vertically partitioned with ISP, where N>4, only vertical mode may be allowed for intra prediction. This ensures that the prediction of any of these sub-partitions (1×N sub-partitions in the case of 4×N coded blocks, where N>8, and 2×N sub-partitions in the case of 4×8 coded blocks) will be independent of the other sub-partitions in the coded block. In this way, the video codec may perform prediction of the sub-partitions in parallel or by treating all sub-partitions as 4×N blocks for prediction. For such coded blocks, intra mode may not be signaled.

[0187] Fig.17 17 is a conceptual diagram illustrating an example in which a video codec is allowed to perform only vertical intra (VER Intra) mode prediction for a 4×N coding block 1702 that the video codec codes using ISP and vertical partitioning. That is, Fig.17 An example of how a vertical partitioning sub-partition of a 4×N coding block 1702 can be predicted using only vertical intra mode is shown. In the illustrative examples, the coding block 1702 can have a height of at least 8-sample size (N>8). In these examples, the coding block 1702 can be divided into four 1×N sub-partitions (e.g., sub-blocks, prediction blocks). In any case, the inverse transform operation is still performed at the 1×N sub-partition level.

[0188] For an 8×N coding block vertically divided by ISP, where N>4, prediction of samples is performed on a prediction block group (PBG) with a width of four samples. For example, when the coding block size is 8×N (N>4), the video codec may identify two PBGs as each having a size of 4×N. Fig.18 An example of how to divide an 8×N subpartition into two prediction regions (PBG1 or region 1 and PBG2 or region 2) of size 4×N each and four transforms of size 2×N each is shown. Fig.18 In the examples of FIG. 1 , the video codec may encode an 8×N (N>4) coding block 1802 with ISP and vertical partitioning. In these examples, the video codec may perform prediction on a 4×N prediction region (or PBG). In addition, the transform (SP) size may be 2×N. In some examples, the prediction of region 2 depends on the reconstruction of region 1. Therefore, the prediction of any sample within the coding block 1802 depends only on the reconstructed samples from neighboring coding blocks and the previous 4×N prediction region.

[0189] exist Fig.17 In a prediction region (prediction region 1 or 2), the size is 1xN and similar to Fig.18 The four rows of residuals in the ISP and the two residual blocks of 2xN can be generated in parallel or together in one step. For example, the vertical inverse transform operations of the two 2xN sub-partitions within each prediction region can be performed by considering the region as a 4xN block, because the vertical transform for all sub-partitions in the ISP is the same. The 2xN sub-partitions can also be horizontally transformed in parallel by processing them separately or by treating the two sub-partitions as a single 4xN block and treating the horizontal inverse transform operation as a combination of two smaller horizontal inverse transform operations.

[0190] Table 6 below summarizes the partition sizes for prediction size and transform size. In some examples, the prediction size and transform size may be different for other coding block sizes. For example, the prediction and transform sizes may be as specified for ISP in VVC WD4.

[0191] Table 6 - Example partition sizes for prediction size and transform size

[0192] Codec block size Split size for prediction Change size 4×8 4×8 (vertical mode only) 2×N 4×N(N>8) 4×N (vertical mode only) 1×N 8×N(N>4) 4×N 2×N

[0193] Changes to the relevant specification parts in the fifth working draft of VVC (hereinafter referred to as "VVC WD5" and available at phenix.it-sudparis.eu / jvet / doc_end_user / documents / 14_Geneva / wg11 / JVET-N1001-v8.zip) are indicated below with change marks. Deleted items are shown with strikethrough, and added items are shown in bold italics.

[0194] In some examples, a video encoder may use the following inputs in order to perform the coding process:

[0195] 1. Sample point position (xTb0, yTb0), which specifies the top left sample point of the current transform block relative to the top left sample point of the current image,

[0196] 2. The variable nTbW specifies the width of the current transform block,

[0197] 3. The variable nTbH specifies the height of the current transform block,

[0198] 4. The variable predModeIntra that specifies the intra prediction mode, and

[0199] 5. Specify the variable cIdx for the color component of the current block.

[0200] The output of this process is the reconstructed image that is modified before in-loop filtering.

[0201] The maximum transform block size maxTbSize is derived using Equation 2 as follows:

[0202] maxTbSize=(cIdx==0)? MaxTbSizeY:MaxTbSizeY / 2(2)

[0203] The luma sample locations are derived using Equation 3 as follows:

[0204] (xTbY,yTbY)=(cIdx==0)? (xTb0,yTb0):(xTb0*2,yTb0*2)(3)

[0205] Depending on maxTbSize, the following applies:

[0206] If IntraSubPartitionsSplitType is equal to NO_ISP_SPLIT and nTbW is greater than maxTbSize or nTbH is greater than maxTbSize, then the following ordered steps (1. to 5.) apply:

[0207] 1. The variables newTbW and newTbH are derived as follows:

[0208] newTbW=(nTbW>maxTbSize)? (nTbW / 2):nTbW and newTbH=(nTbH>maxTbSize)? (nTbH / 2):nTbH.

[0209] 2. The general decoding process for intra blocks as specified in that clause is called with position (xTb0, yTb0), transform block width nTbW set equal to newTbW and height nTbH set equal to newTbH, intra prediction mode predModeIntra, and variable cIdx as input, and the output is the modified reconstructed image before in-loop filtering.

[0210] 3. If nTbW is greater than maxTbSize, then the general decoding process for intra blocks as specified in that clause is called with the position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0), the transform block width nTbW set equal to newTbW and the height nTbH set equal to newTbH, the intra prediction mode predModeIntra, and the variable cIdx as input, and the output is the modified reconstructed image before in-loop filtering.

[0211] 4. If nTbH is greater than maxTbSize, then the general decoding process for intra blocks as specified in that clause is called with the position (xTb0, yTb0) set equal to (xTb0, yTb0+newTbH), the transform block width nTbW set equal to newTbW and the height nTbH set equal to newTbH, the intra prediction mode predModeIntra, and the variable cIdx as input, and the output is the modified reconstructed image before in-loop filtering.

[0212] 5. If nTbW is greater than maxTbSize and nTbH is greater than maxTbSize, then the general decoding process for intra blocks as specified in that clause is called with the position (xTb0, yTb0) set equal to (xTb0+newTbW, yTb0+newTbH), the transform block width nTbW set equal to newTbW and the height nTbH set equal to newTbH, the intra prediction mode predModeIntraMode, and the variable cIdx as input, and the output is the modified reconstructed image before in-loop filtering.

[0213] Otherwise, the following ordered steps (1. and 2.) apply. For example, when the video codec determines that IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT (e.g., when IntraSubPartitionsSplitType is equal to ISP_VER_SPLIT or ISP_HOR_SPLIT) or when both nTbW and nTbH are less than or equal to maxTbSize, the video codec may derive the following variables.

[0214] 1. In some examples, the video codec may derive multiple variables, including nW, nH, xPartIncrement, yPartIncrement, nPbW, and pbFactor. For example, the video codec may derive such variables using equations (6) to (11), as follows:

[0215] nW=IntraSubPartitionsSplitType==ISP_VER_SPLIT?

[0216] nTbW / NumIntraSubPartitions:nTbW(6)

[0217] nH=IntraSubPartitionsSplitType==ISP_HOR_SPLIT?

[0218] nTbH / NumIntraSubPartitions:nTbH(7)

[0219] xPartIncrement=ISP_VER_SPLIT? 1;0(8)

[0220] yPartIncrement=ISP_HOR_SPLIT? 1;0(9)

[0221] nPbW=max(4,nW)(10)

[0222] pbFactor=nPbW / nW(11)

[0223] 2. Additionally, set the variables xPartIdx and yPartIdx equal to 0.

[0224] In such an example, the video codec may apply the following ordered steps (1. to 4.) for i=0..NumIntraSubPartitions-1, in order:

[0225] 1. The variables xPartIdx, yPartIdx, and xPartPbIdx are updated as follows:

[0226] xPartIdx=xPartIdx+xPartIncrement(12)

[0227] yPartIdx=yPartIdx+yPartIncrement(13)

[0228] xPartPbIdx=xPartIdx%pbFactor(14)

[0229] 2. In some examples, when xPartPbIdx is equal to 0, the video codec may invoke the intra sample prediction process as specified in clause 8.4.5.2. In such examples, the video codec may invoke the intra sample prediction process with the position (xTbCmp, yTbCmp) set equal to (xTb0+nW*xPartIdx, yTb0+nH*yPartIdx), the intra prediction mode predModeIntra, the transform block width nTbW and height nTbH set equal to nPbW and nH, the coding block width nCbW set equal to nTbW, and the coding block height nCbH set equal to nTbH, and the variable cIdx as input, and the output is an (nPbW)x(nH) array predSamples (e.g., a prediction sample array).

[0230] 3. Invoke the scaling and transform process as specified in clause 8.7.2 with as input the luma position (xTbY, yTbY) set equal to (xTbY+nW*xPartIdx, yTbY+nH*yPartIdx), the variable cIdx, the transform width nTbW and transform height nTbH set equal to nW and nH, and the output is the (nPbW)x(nH) array resSamples (e.g., the residual sample array). In one example, the output may be a (nW)×(nH) array resSamples.

[0231] 4. Invoke the image reconstruction process for color components as specified in clause 8.7.5, with the transform block position (xTbComp, yTbComp) set equal to (xTb0+nW*xPartIdx, yTb0+nH*yPartIdx), the transform block width nTbW set equal to nW and nH, the transform block height nTbH, the variable cIdx, the (nW)×(nH) array predSamples[x][y] (where x=xPartPbIdx*nW..(xPartPbIdx+1)*nW–1, y=0..nH–1) and the (nW)×(nH) array resSamples as input, and the output is the modified reconstructed image before in-loop filtering.

[0232] In some examples, for some block sizes, the following changes may restrict the intra prediction mode to vertical intra mode. Example changes are made to the signaling in section 7.3.7.5 of VVC WD5. Deleted items are shown in strikethrough, and added items are shown in italics.

[0233] Table 7 - Restricted prediction mode

[0234]

[0235]

[0236]

[0237]

[0238]

[0239] Exemplary changes are made to the derivation of luma intra prediction mode in section 8.4.3 of VVC WD5. Deleted items are shown in strikethrough and added items are shown in italics.

[0240] For example, the video codec may perform a luma intra prediction mode derivation process. The inputs to this process include: (1) a luma position (xCb, yCb) that specifies the top left sample of the current luma coded block relative to the top left luma sample of the current image, (2) a variable cbWidth that specifies the width of the current coded block in luma samples, and / or (3) a variable cbHeight that specifies the height of the current coded block in luma samples.

[0241] In this process, based on the above input, the video codec can derive the luma intra prediction mode IntraPredModeY[xCb][yCb].

[0242] Table 8 specifies the values ​​of the intra prediction modes IntraPredModeY[xCb][yCb] and the associated names. It should be noted that the intra prediction modes INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM may be applicable only to chroma components.

[0243] Table 8 - Specification of intra prediction modes and associated names

[0244] Intra prediction mode Associated name 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM,INTRA_L_CCLM,INTRA_T_CCLM

[0245] IntraPredModeY[xCb][yCb] is derived as follows: If BdpcmFlag[xCb][yCb] is equal to 1 or intra_luma_not_planar_flag[xCb][yCb] is equal to 0, then IntraPredModeY[xCb][yCb] is set equal to INTRA_PLANAR. Otherwise, if IntraSubPartitionsSplitType is equal to ISP_VER_SPLIT and cbWidth is equal to 4, then IntraPredmodeY[xCb][yCb] is set equal to INTRA_ANGULAR50. Otherwise (e.g., intra_luma_not_planar_flag[xCb][yCb] is equal to 1), the neighboring positions (xNbA, yNbA) and (xNbB, yNbB) are set equal to (xCb-1, yCb+cbHeight-1) and (xCb+cbWidth-1, yCb-1), respectively.

[0246] Fig.19A and 19B1 is a conceptual diagram illustrating an exemplary QTBT structure 130 and a corresponding coding tree unit (CTU) 132. Solid lines represent quadtree partitions, and dashed lines indicate binary tree partitions. In each partition (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which partition type (e.g., horizontal, vertical, etc.) is used. In an illustrative and non-limiting example, a value of 0 may indicate horizontal partitioning, while a value of 1 may indicate vertical partitioning. In another example involving quadtree partitioning, it may not be necessary to indicate the partition type because the quadtree node generally divides the block horizontally and vertically into 4 sub-blocks (e.g., prediction blocks). In such an example, the sub-blocks may or may have equal sizes. Therefore, the video encoder 200 may encode and the video decoder 300 may decode the syntax elements (e.g., partition information) of the regional tree level (i.e., the first level) (i.e., the solid line) of the QTBT structure 130 and the syntax elements (e.g., partition information) of the prediction tree level (i.e., the second level) (i.e., the dashed line) of the QTBT structure 130. For a CU represented by a terminal leaf node of the QTBT structure 130 , the video encoder 200 may encode video data (such as prediction and transform data), and the video decoder 300 may decode the same.

[0247] Generally speaking, Fig.19B The CTU 132 may be associated with parameters that define the size of blocks corresponding to nodes of the first and second level QTBT structures 130. These parameters may include CTU size (indicating the size of the CTU 132 in samples), minimum quadtree size (MinQTSize, indicating the minimum allowed quadtree leaf node size), maximum binary tree size (MaxBTSize, indicating the maximum allowed binary tree root node size), maximum binary tree depth (MaxBTDepth, indicating the maximum allowed binary tree depth), and minimum binary tree size (MinBTSize, indicating the minimum allowed binary tree leaf node size).

[0248] The root node of the QTBT structure corresponding to the CTU can have four child nodes at the first level of the QTBT structure, and each child node can be segmented according to the quadtree segmentation. That is, the node of the first level is a leaf node (no child node) or has four child nodes. The example of the QTBT structure 130 represents such a node, which contains child nodes and parent nodes with solid line branches. If the node of the first level is not larger than the maximum allowed binary tree root node size (MaxBTSize), the node can be further segmented by the respective binary trees. The binary tree partitioning of a node can be iterated until the node generated by the partition reaches the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). The example of the QTBT structure 130 represents such a node as having a dotted branch. The binary tree leaf node is called a coding unit (CU), which is used for prediction (e.g., intra-image or inter-image prediction) and transformation without any further segmentation. As described above, a CU may also be referred to as a "video block", "coding block" or "block".

[0249] In one example of a QTBT partitioning structure, the CTU size is set to 128×128 (luminance sample and two corresponding 64×64 chrominance samples), MinQTSize is set to 16×16, MaxBTSize is set to 64×64, MinBTSize (for width and height) is set to 4, and MaxBTDepth is set to 4. First, quadtree partitioning is applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can have sizes from 16×16 (ie, MinQTSize) to 128×128 (ie, CTU size). If the leaf quadtree node is 128×128, since its size exceeds MaxBTSize (64×64 in this example), the leaf quadtree node will not be further divided by the binary tree. Otherwise, the leaf quadtree node will be further divided by the binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree and has a binary tree depth of 0. When the binary tree depth reaches MaxBTDepth (4 in this example), no further division is allowed. When the width of the binary tree node is equal to MinBTSize (4 in this example), this means that no further vertical division is allowed. Similarly, a binary tree node with a height equal to MinBTSize indicates that no further horizontal division is allowed for the binary tree node. As described above, the leaf nodes of the binary tree are called CUs and are further processed according to prediction and transformation without further segmentation.

[0250] Fig. 20 is a block diagram illustrating an example video encoder 200 that may perform the techniques of this disclosure. Fig. 20For purposes of explanation and should not be considered a constraint on the techniques extensively exemplified and described in this disclosure. For purposes of explanation, this disclosure describes the video encoder 200 in the context of video codec standards such as the HEVC video codec standard and the developing H.266 video codec standard. However, the techniques of this disclosure are not limited to these video codec standards and are generally applicable to video encoding and decoding.

[0251] exist Fig. 20 In the example of , the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filter unit 216, the DPB 218, and the entropy coding unit 220 may be implemented in one or more processors or processing circuits. In addition, the video encoder 200 may include additional or alternative processors or processing circuits to perform these and other functions.

[0252] The video data memory 230 may store video data to be encoded by the components of the video encoder 200. The video encoder 200 may receive, for example, video source 104 ( Figure 1 ) receives video data stored in video data memory 230. DPB 218 can be used as a reference image memory, which stores reference video data for use by video encoder 200 when predicting subsequent video data. Video data memory 230 and DPB 218 can be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM) or other types of memory devices. Video data memory 230 and DPB 218 can be provided by the same memory device or a separate memory device. In various examples, video data memory 230 can be placed on-chip with other components of video encoder 200, as shown, or placed off-chip relative to those components.

[0253] In the present disclosure, references to the video data memory 230 should not be interpreted as limited to a memory inside the video encoder 200 (unless specifically stated otherwise) or a memory outside the video encoder 200 (unless specifically stated otherwise). Of course, references to the video data memory 230 should be understood as a reference memory that stores video data received by the video encoder 200 for encoding (e.g., video data to be encoded of a current block). Figure 1 The memory 106 can also provide temporary storage for the outputs of various units of the video encoder 200 .

[0254] Graphic Fig. 20 The various units are used to help understand the operations performed by the video encoder 200. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and preset operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, to receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more units may be integrated circuits.

[0255] The video encoder 200 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), a digital circuit, an analog circuit, and / or a programmable core formed by programmable circuits. In examples where the operation of the video encoder 200 is performed using software executed by programmable circuits, the memory 106 ( Figure 1 ) may store object code for software received and executed by the video encoder 200, or another memory (not shown) within the video encoder 200 may store such instructions.

[0256] The video data memory 230 is configured to store received video data. The video encoder 200 may retrieve images of video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 may be original video data to be encoded.

[0257] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units for performing video prediction according to other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0258] Generally speaking, the mode selection unit 202 coordinates multiple encoding passes to test combinations of encoding parameters and derive rate-distortion values ​​for such combinations. The encoding parameters may include the partitioning of CTUs into CUs, the prediction mode of the CUs, the transform type of the residual data of the CUs, the quantization parameter of the residual data of the CUs, etc. The mode selection unit 202 may ultimately select a combination of encoding parameters that has a better rate-distortion value than other tested combinations.

[0259] The video encoder 200 may divide the image obtained from the video data memory 230 into a series of CTUs and encapsulate one or more CTUs in a slice. The mode selection unit 202 may divide the CTU of the image according to a tree structure (such as the QTBT structure or quadtree structure of the HEVC described above). As described above, the video encoder 200 may form one or more CUs by dividing the CTU according to the tree structure. Such a CU may also be generally referred to as a "video block" or "block".

[0260] In general, the mode selection unit 202 also controls its components (e.g., the motion estimation unit 222, the motion compensation unit 224, and the intra prediction unit 226) to generate a prediction block for the current block (e.g., the overlapping portion of the current CU or PU and TU in HEVC). In order to inter-predict the current block, the motion estimation unit 222 can perform a motion search to identify one or more closely matching reference blocks in one or more reference images (e.g., one or more previously encoded images stored in the DPB 218). In particular, the motion estimation unit 222 can calculate values ​​indicating how similar the potential reference blocks are to the current block based on, for example, the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), etc. The motion estimation unit 222 can generally perform these calculations using the sample-by-sample differences between the current block and the reference block under consideration. The motion estimation unit 222 can identify the reference block having the lowest value generated by these calculations, indicating the reference block that most closely matches the current block.

[0261] The motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference image relative to a current block in a current image. The motion estimation unit 222 may then provide the motion vectors to the motion compensation unit 224. For example, for unidirectional inter prediction, the motion estimation unit 222 may provide a single motion vector, while for bidirectional inter prediction, the motion estimation unit 222 may provide two motion vectors. The motion compensation unit 224 may then use the motion vectors to generate a prediction block. For example, the motion compensation unit 224 may use the motion vectors to retrieve data for the reference block. As another example, if the motion vectors have fractional sample precision, the motion compensation unit 224 may interpolate the prediction block according to one or more interpolation filters. Additionally, for bidirectional inter prediction, the motion compensation unit 224 may retrieve data for two reference blocks identified by respective motion vectors and combine the retrieved data (e.g., by sample-by-sample averaging or weighted averaging).

[0262] As another example, for intra prediction or intra prediction coding, the intra prediction unit 226 can generate a prediction block based on samples adjacent to the current block. For example, for directional mode, the intra prediction unit 226 can generally mathematically combine the values ​​of adjacent samples and fill these calculated values ​​along a defined direction on the current block to generate a prediction block. As another example, for DC mode, the intra prediction unit 226 can calculate the average value of adjacent samples of the current block and generate a prediction block to contain the average value obtained for each sample of the prediction block.

[0263] The mode selection unit 202 provides the prediction block to the residual generation unit 204. The residual generation unit 204 receives the original uncoded version of the current block from the video data memory 230 and receives the prediction block from the mode selection unit 202. The residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines a residual block for the current block. In some examples, the residual generation unit 204 may also use residual differential pulse code modulation (RDPCM) to determine the difference between the sample values ​​in the residual block to generate the residual block. In some examples, the residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0264] In the example where the mode selection unit 202 partitions the CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video encoder 200 and the video decoder 300 may support PUs of various sizes. As described above, the size of the CU may refer to the size of the luma coding block of the CU, and the size of the PU may refer to the size of the luma prediction unit of the PU. Assuming that the size of a particular CU is 2N×2N, the video encoder 200 may support PU sizes of 2N×2N or N×N for intra-frame prediction, and symmetric PU sizes of 2N×2N, 2N×N, N×2N, N×N or similar for inter-frame prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitioning of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N.

[0265] In an example where the mode selection unit does not further split the CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As described above, the size of a CU may refer to the size of the luma coding block of the CU. The video encoder 200 and the video decoder 300 may support a CU size of 2N×2N, 2N×N, or N×2N.

[0266] For other video coding techniques, such as intra block copy mode coding, affine mode coding, and linear model (LM) mode coding as some examples, the mode selection unit 202 generates a prediction block for the current block being encoded via the respective units associated with the coding technique. In some examples, such as palette mode coding, the mode selection unit 202 may not generate a prediction block, but instead generate syntax elements indicating how to reconstruct the block based on the selected palette. In such a mode, the mode selection unit 202 may provide these syntax elements to the entropy coding unit 220 for encoding.

[0267] As described above, the residual generation unit 204 receives video data of a current block and a corresponding prediction block. Then, the residual generation unit 204 generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0268] The transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 can apply various transforms to the residual block to form a transform coefficient block. For example, the transform processing unit 206 can apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, the transform processing unit 206 can perform multiple transforms on the residual block, for example, a primary transform and a secondary transform such as a rotation transform. In some examples, the transform processing unit 206 does not apply a transform to the residual block.

[0269] The quantization unit 208 may quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. The quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (QP) value associated with the current block. The video encoder 200 (e.g., via the mode selection unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce information loss, and thus, the quantized transform coefficients may have lower precision than the original transform coefficients produced by the transform processing unit 206.

[0270] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block from the transform coefficient block. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although potentially with some degree of distortion) based on the reconstructed residual block and the prediction block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples of the prediction block generated by the mode selection unit 202 to generate a reconstructed block.

[0271] Filter unit 216 may perform one or more filter operations on the reconstructed block. For example, filter unit 216 may perform a deblocking operation to reduce blocking artifacts along the edge of the CU. In some examples, the operation of filter unit 216 may be skipped.

[0272] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in an example where the operation of the filter unit 216 is not required, the reconstruction unit 214 can store the reconstructed blocks to the DPB 218. In an example where the operation of the filter unit 216 is required, the filter unit 216 can store the filtered reconstructed blocks to the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can obtain a reference image from the DPB 218, which is formed by the reconstructed (and potentially filtered) blocks, to perform inter-frame prediction on blocks of subsequently encoded images. In addition, the intra-frame prediction unit 226 can use the reconstructed blocks in the DPB 218 of the current image to perform intra-frame prediction on other blocks in the current image.

[0273] In general, the entropy coding unit 220 may entropy encode syntax elements received from other functional components of the video encoder 200. For example, the entropy coding unit 220 may entropy encode the quantized transform coefficient blocks from the quantization unit 208. As another example, the entropy coding unit 220 may entropy encode the prediction syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from the mode selection unit 202. The entropy coding unit 220 may perform one or more entropy coding operations on syntax elements of another example of video data to generate entropy encoded data. For example, the entropy coding unit 220 may perform a context adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable to variable (V2V) length coding operation, a syntax-based context adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential-Grundberg coding operation, or another type of entropy coding operation on the data. In some examples, the entropy coding unit 220 may operate in a bypass mode when the syntax elements are not entropy encoded.

[0274] The video encoder 200 may output a bitstream containing syntax elements of entropy coding required to reconstruct a block of a slice or an image. Specifically, the entropy coding unit 220 may output a bitstream.

[0275] The above operations are described for blocks. Such description should be understood as operations for luma coding blocks and / or chroma coding blocks. As mentioned above, in some examples, the luma coding block and the chroma coding block are the luma and chroma components of the CU. In some examples, the luma coding block and the chroma coding block are the luma and chroma components of the PU.

[0276] In some examples, the operations performed for the luma coding block do not need to be repeated for the chroma coding block. As an example, the operations for identifying the motion vector (MV) and reference image for the luma coding block do not need to be repeated to identify the MV and reference image for the chroma block. Instead, the MV of the luma coding block can be scaled to determine the MV of the chroma block, and the reference image can be the same. As another example, the intra prediction process can be the same for the luma coding block and the chroma coding block.

[0277] The video encoder 200 may be configured to perform any of the prediction block grouping techniques described in the present disclosure. For example, the intra prediction unit 226 may perform ISP coding to partition a coding unit (CU) into sub-blocks (e.g., prediction blocks). In addition, the intra prediction unit 226 may group sub-blocks into prediction block groups. In this way, the video encoder 200 may be viewed as including one or more processors implemented in a circuit and configured to perform partitioning (e.g., vertical partitioning) of a coding unit (CU) of video data using intra sub-partitioning (ISP) to form a prediction block set. The prediction block may include at least a first prediction block and a second prediction block. The video encoder 200 may then group multiple prediction blocks in the prediction block set into a first prediction block group (PBG). In addition, the video encoder 200 may reconstruct the samples of the prediction block included in the first PBG independently of the samples of other prediction blocks included in the first PBG.

[0278] Fig.21 is a block diagram illustrating an example video decoder 300 that may perform the techniques of this disclosure. Fig.21 The techniques widely exemplified and described in this disclosure are for the purpose of explanation, not limitation. For the purpose of illustration, this disclosure describes a video decoder 300 according to the technical descriptions of JEM, VCC, and HEVC. However, the techniques of this disclosure may be performed by a video codec device configured for other video codec standards.

[0279] exist Fig.21 In the example of , the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 314. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, the filter unit 312, and the DPB 314 may be implemented in one or more processors or processing circuits. In addition, the video decoder 300 may include additional or alternative processors or processing circuits to perform these and other functions.

[0280] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 may include additional units to perform predictions according to other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0281] CPB memory 320 may store video data, such as an encoded video bitstream, to be decoded by components of video decoder 300. For example, the video bitstream may be obtained from computer readable medium 110 ( Figure 1 ) obtains video data stored in CPB memory 320. CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from an encoded video bitstream. Moreover, CPB memory 320 may store video data other than syntax elements of a coded image, such as temporary data representing outputs from various units of video decoder 300. Generally speaking, DPB 314 stores decoded images, and video decoder 300 may output decoded images and / or use them as reference video data when decoding subsequent data or images of the encoded video bitstream. CPB memory 320 and DPB 314 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or by separate memory devices. In various examples, CPB memory 320 may be placed on-chip with other components of video decoder 300, or off-chip relative to those components.

[0282] Additionally or alternatively, in some examples, video decoder 300 may retrieve the video from memory 120 ( Figure 1 ) to obtain the encoded video data. That is, the memory 120 can store data together with the CPB memory 320 as discussed above. Similarly, when some or all of the functions of the video decoder 300 are implemented in software to be executed by the processing circuit of the video decoder 300, the memory 120 can store instructions to be executed by the video decoder 300.

[0283] Graphics Fig.21 Various units are shown to aid in understanding the operations performed by the video decoder 300. These units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Fig. 20, fixed-function circuits refer to circuits that provide specific functions and are preset in the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, to receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more units may be integrated circuits.

[0284] The video decoder 300 may include an ALU, an EFU, a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example where the operation of the video decoder 300 is performed by software executed on the programmable circuit, an on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0285] The entropy decoding unit 302 may receive the encoded video data from the CPB and entropy decode the video data to reproduce the syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filter unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0286] Generally speaking, the video decoder 300 reconstructs an image on a block-by-block basis. The video decoder 300 may perform a reconstruction operation on each block individually (where a block currently being reconstructed (ie, decoded) may be referred to as a "current block").

[0287] The entropy decoding unit 302 may entropy decode syntax elements defining the quantized transform coefficients of the quantized transform coefficient block and transform information such as a quantization parameter (QP) and / or (multiple) transform mode indications. The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine a degree of quantization, and likewise, determine a degree of inverse quantization for the inverse quantization unit 306 to apply. The inverse quantization unit 306 may inverse quantize the quantized transform coefficients (e.g., performing a bitwise left shift operation). The inverse quantization unit 306 may thereby form a transform coefficient block containing the transform coefficients.

[0288] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the transform coefficient block.

[0289] Further, prediction processing unit 304 generates a prediction block based on the prediction information syntax element entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, motion compensation unit 316 may generate the prediction block. In this case, the prediction information syntax element may indicate a reference picture in DPB 314 from which to obtain the reference block, and a motion vector that identifies the position of the reference block in the reference picture relative to the current block in the current picture. Motion compensation unit 316 may generally generate a prediction block in the same manner as for motion compensation unit 224 ( Fig. 20 ) is performed in a manner substantially similar to that described herein.

[0290] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra-prediction unit 318 may generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Again, the intra-prediction unit 318 may generally generate a prediction block in the same manner as for the intra-prediction unit 226 ( Fig. 20 The intra prediction process is performed in a manner substantially similar to that described in the foregoing description. The intra prediction unit 318 may obtain data of neighboring samples of the current block from the DPB 314.

[0291] The reconstruction unit 310 may reconstruct the current block using the prediction block and the residual block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

[0292] The filter unit 312 may perform one or more filter operations on the reconstructed block. For example, the filter unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operations of the filter unit 312 may not necessarily be performed in all examples.

[0293] The video decoder 300 may store the reconstructed block in the DPB 314. For example, in an example where the operation of the filter unit 312 is not performed, the reconstruction unit 310 may store the reconstructed block in the DPB 314. In an example where the operation of the filter unit 312 is performed, the filter unit 312 may store the filtered reconstructed block in the DPB 314. As described above, the DPB 314 may provide reference information to the prediction processing unit 304, such as samples of the current image for intra-frame prediction and previously decoded images for subsequent motion compensation. In addition, the video decoder 300 may output a decoded image from the DPB 314 for subsequent presentation on, for example, Figure 1 On the display device 118 of the display device.

[0294] The video decoder 300 may be configured to perform any of the prediction block grouping techniques described in the present disclosure. For example, the intra prediction unit 318 may perform ISP coding to partition a coding unit (CU) into sub-blocks (e.g., prediction blocks). In addition, the intra prediction unit 318 may group the sub-blocks into prediction block groups. In this way, the video decoder 300 may be viewed as including one or more processors implemented in a circuit and configured to perform partitioning (e.g., vertical partitioning) of a coding unit (CU) of video data using intra sub-partitioning (ISP) to form a prediction block set. The prediction block may include at least a first prediction block and a second prediction block. The video decoder 300 may group multiple prediction blocks of the prediction block set into a first prediction block group (PBG). In addition, the video decoder 300 may reconstruct the samples of the prediction block included in the first PBG independently of the samples of other prediction blocks included in the first PBG.

[0295] Fig. 22 2 is a flowchart illustrating an example method for encoding a current block. The current block may include a current CU. Although with respect to the video encoder 200 ( Figure 1 and 20 ), but it should be understood that other devices may be configured to perform similar Fig. 22 method.

[0296] In this example, the video encoder 200 initially predicts the current block (350). For example, the video encoder 200 may form a prediction block for the current block. Then, the video encoder 200 may calculate a residual block for the current block (352). That is, the video encoder 200 may calculate luma residual data and chroma residual data for the current block. To calculate the residual block, the video encoder 200 may calculate the difference between the original uncoded block and the prediction block for the current block. Then, the video encoder 200 may transform and quantize the coefficients of the residual block (354). Next, the video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or after the scan, the video encoder 200 may entropy encode the transform coefficients (358). For example, the video encoder 200 may encode the transform coefficients using CAVLC or CABAC. The video encoder 200 may then output the entropy encoded data of the block (360). The video encoder 200 may implement a reconstruction loop. For example, video encoder 200 may implement a reconstruction loop to reconstruct coded video data using the prediction block grouping techniques described in this disclosure.

[0297] Fig.23 is a flowchart illustrating an example method for decoding a current block of video data. The current block may include a current CU. Although with respect to the video decoder 300 ( Figure 1 and 21 ), but it should be understood that other devices may be configured to perform similar Fig.23 method.

[0298] The video decoder 300 may receive entropy coded data of a current block, such as entropy coded prediction information and entropy coded data of transform coefficients of a residual block corresponding to the current block (370). The video decoder 300 may decode the entropy coded data to determine the prediction information for the current block and reproduce the transform coefficients of the residual block (372). The video decoder 300 may predict the current block (374), for example, using an intra-frame or inter-frame prediction mode indicated by the prediction information of the current block to calculate a prediction block for the current block. The video decoder 300 may then inverse scan the reproduced transform coefficients (376) to create a block of quantized transform coefficients. The video decoder 300 may then inverse quantize and inverse transform the coefficients to produce a residual block (378). The video decoder 300 may finally decode the current block (380) by combining the prediction block and the residual block. The video decoder 300 may reconstruct the samples of the current block using the prediction block grouping techniques described in the present disclosure.

[0299] Fig.242 is a flowchart illustrating an example method for performing prediction block grouping on a current block of video data. The current block may include a current CU. Although with respect to the video encoder 200 ( Figure 1 and 20 ), but it should be understood that other devices may be configured to perform similar Fig.24 method.

[0300] In some examples, a video codec (e.g., video encoder 200 and / or video decoder 300) may perform partitioning (e.g., vertical partitioning) of a coding unit (CU) of video data using intra-frame sub-partitioning (ISP). For example, the video codec may perform vertical and / or horizontal partitioning on a first CU to form a prediction block set (2402). In an illustrative example, the video codec may perform vertical partitioning of the first CU to form a prediction block set. In any case, the prediction block set may include all prediction blocks of the CU or only a subset of the prediction blocks of the CU. That is, the video codec may form a prediction block set that does not necessarily include all prediction blocks of the CU. For example, the video codec may perform partitioning of the first CU to form a first prediction block set and one or more other prediction block sets. The one or more other prediction block sets may be mutually exclusive of each other and of the first prediction block set. In some examples, the prediction block sets may overlap with each other to some extent, so that a portion of the CU may be partitioned into more than one prediction block. Although various techniques of this disclosure describe a video codec that partitions a CU into prediction block sets, the techniques of this disclosure are not limited thereto, and the video codec may partition a CU into multiple prediction block sets.

[0301] In some examples, the prediction blocks in the prediction block set may include at least a first prediction block and a second prediction block (e.g., at least two vertical prediction blocks). In some examples, the first prediction block includes a sample width less than or equal to two. For example, the first prediction block may have a sample width of 1 or 2. In addition, the second prediction block may have a sample width of 1 or 2. In some examples, the first prediction block and the second prediction block may have the same width. For example, the first prediction block and the second prediction block may both have a sample width of 2. However, in some examples, the first prediction block and the second prediction block may have different widths, such as the first prediction block having a width of 2 and the second prediction block having a width of 1. In such an example, the third prediction block may be included in the first PBG so that the first PBG may have a width of at least 4 sample sizes.

[0302] In some examples, the video codec may group a plurality of prediction blocks from the prediction block set into a first prediction block group (PBG) (2404). In one example, the video codec may group one or more prediction blocks from the first prediction block set into the first PBG. For example, the video codec may group a first prediction block and a second prediction block from the first prediction block set into the first PBG. In some examples, the first PBG may include only one prediction block from the first prediction block set.

[0303] In some examples, the video codec may group a first prediction block set of a CU to form one or more PBGs. In these examples, the video codec may group the PBGs similarly or differently relative to one or more other prediction block sets of the same CU. For example, the video codec may group at least one vertical prediction block from a first prediction block set and at least one horizontal prediction block from the same prediction block set into a first PBG. In these examples, the video codec may group at least one vertical prediction block from a second prediction block set and at least one horizontal prediction block from a second prediction block set into a second PBG. However, in some examples, the video codec may instead group only vertical prediction blocks from a second prediction block set and / or only horizontal prediction blocks from a second prediction block set into a second PBG, wherein the first prediction block set includes only vertical sub-blocks, only horizontal sub-blocks, a mixture of vertical and horizontal blocks, etc., and the second prediction block set includes only vertical sub-blocks, only horizontal sub-blocks, a mixture of vertical and horizontal blocks, etc. In some embodiments, the video codec groups only specific types of prediction blocks into PBCs. In a non-limiting example involving a CU partitioned using multiple partition types (e.g., vertical and horizontal partitioning of a single CU), the video codec may group only those prediction blocks that are the result of the vertical partitioning portion of the overall partitioning scheme into PBGs. Similarly, the video codec may forgo grouping prediction blocks that the video codec partitioned in a different manner (e.g., using horizontal partitioning). In another example, the video codec may form a group that includes a vertical prediction block and one or more horizontal prediction blocks adjacent to the vertical prediction block. The video codec may additionally form a group that includes any remaining prediction blocks (e.g., non-adjacent prediction blocks), or may determine not to form groups for certain prediction blocks in one or more prediction block sets for the CU. Furthermore, the video codec may group only prediction blocks of a particular size (e.g., a size that satisfies a predetermined size threshold).

[0304] In some examples, a PBG may be specified as having a minimum dimension of nW samples. For example, a PBG may have a minimum width of four samples (e.g., nW=4). That is, the first PBG may have a sample size width of at least four. In such a case, if the dimension size of a subblock is less than nW, then the adjacent one or more prediction blocks may be considered as a PBG with a dimension size of nW samples. The dimension size may be applied to height, width, or a function derived from height and width (e.g., width*height).

[0305] The video codec may reconstruct samples of a prediction block contained in the first PBG independently of samples of other prediction blocks contained in the first PBG (2406). When the video codec reconstructs a sample from a sub-block without accessing, reading, or performing other memory access processes to identify or reconstruct samples in another sub-block within the same PBG, the video codec may reconstruct samples of the prediction block independently of other prediction blocks within the PBG. As a corollary, when the video codec performs sequential operations, the video codec may reconstruct samples from one group based on reconstruction of samples from another group. That is, the sequential operation involves first reconstructing one or more samples from the first group, and then reconstructing one or more samples from the other group based on the samples from the first group. In some cases, independent reconstruction may involve a parallel process, wherein when the video codec reconstructs samples from a second prediction block in the same first PBG, the video codec reconstructs samples of a first prediction block from the first PBG in parallel or substantially simultaneously (e.g., without necessary temporal delay) with reconstructing samples of a second prediction block from the same first PBG.

[0306] In examples where the first PBG includes a first prediction block and a second prediction block, the video codec may reconstruct samples of the first prediction block independently of samples of the second prediction block. In some examples, when certain filtering operations are applied to the prediction block that may result in correlations between prediction values ​​of samples within the PBG, restrictions on prediction correlations of samples within the PBG may still be considered valid as long as conventional intra prediction (without post-prediction filtering operations) does not result in correlations. For example, restrictions may not apply to correlations due to PDPC, boundary filtering, or other similar operations.

[0307] In some examples, the video codec may form a second PBG such that a single CU contains at least two PBGs. For example, the video codec may group one or more prediction blocks into a first PBG. Similarly, the video codec may group a second set of prediction blocks into a second PBG. The prediction blocks of each PBG may contain a non-overlapping set of prediction blocks. That is, the prediction blocks of one PBG may be mutually exclusive with the prediction blocks of another PBG. In any case, the video codec may reconstruct the samples of the prediction blocks contained in the second PBG independently of the samples of other prediction blocks contained in the second PBG.

[0308] In addition, in some examples, the video codec may reconstruct samples of a prediction block included in a second PBG based on samples of a prediction block included in a first PBG. That is, reconstruction of samples in one PBG may depend on reconstruction of samples in another PBG. In these examples, the video codec may still maintain prediction independence of samples within a single PBG. In some examples, prediction of samples included in the second PBG may not be based on a prediction block included in the first PBG, but may be based on a prediction block included in another CU (e.g., a neighboring CU).

[0309] In some examples, the video codec may partition the CU into one or more transform blocks. In these examples, the size of the transform block may be equal to the size of the first prediction block or any prediction block of the CU. In some examples, the size of the transform block may be smaller than the size of the smallest prediction block included in the first PBG. In addition, the size of the transform block may be smaller than the size of the aggregate total size of the prediction blocks included in the first PBG. In another example, the transform block may have a size equal to the size of the PBG. In any case, the first transform block associated with the first PBG may have a size equal to the size of the first prediction block or the second prediction block included in the first PBG.

[0310] In some examples, the video codec may generate an array of prediction samples having a width equal to a combined width of at least the first prediction block and the second prediction block. For example, the video codec may output an array of size (nPbW)×(nH). In some examples, when reconstructing samples of a prediction block, the video codec may generate an array of prediction samples having a width equal to a combined width of at least the first prediction block and the second prediction block.

[0311] In some examples, the video codec may determine the number of prediction block groups based on the width of the coding unit and the width of the first prediction block. The video codec may code the CU according to the number of prediction block groups so that the CU includes the number of prediction block groups. The number of prediction block groups may be referred to as pbFactor (e.g., prediction block group factor) here. In this instance, the video codec may calculate the pbFactor by dividing the width of the CU by the width of the prediction block group. The width of the PBG may be predetermined to have a width of at least 4 sample sizes.

[0312] In some examples, the video codec may calculate pbFactor by dividing the subblock width (e.g., the prediction block width) by the width of the PBG (e.g., the minimum width of the PBG). For example, when the subblock has a width of 1 or 2, the video codec may calculate pbFactor by dividing the subblock width by the width of the PBG, such as by dividing by four when the PBG has a sample width of four. In another example, when the width of the CU is 8 sample size and the prediction block group is 4 sample size, the video codec may calculate pbFactor equal to 2. That is, the number or quantity of PBGs of the CU may be equal to 2 PBGs.

[0313] In some examples, when the width of the prediction block is four or greater, the prediction block width and the PBG width may have the same width. For example, the video codec may group prediction blocks of width four into PBGs of width four. That is, the video codec may group prediction blocks of width X into single PBGs of the same width, where X is greater than or equal to a predefined threshold number. In a non-limiting and illustrative example, the predefined threshold number may be four. In another illustrative example, the video codec may group a first prediction block of width four and a second prediction block of width eight into a single PBG of width eight. In such an example, the video codec may include only a portion of the second prediction block into the PBG group, such as by further partitioning the second prediction block to achieve a specific PBG group size. Although various techniques of the present disclosure discuss width size in certain contexts, the techniques of the present disclosure are not limited thereto, and it will be appreciated that other constraints and / or considerations may be similarly applied to grouping prediction blocks into PBGs. For example, a PBG may have a specific height limit (e.g., a height of four sample sizes), or in another example, a combination of height and width constraints and / or considerations may similarly apply to grouping prediction blocks into PBGs (e.g., a PBG may have a minimum H×W size of M×N, where M and N may not necessarily be equal.

[0314] As mentioned above, the Fig.24The exemplary method of can be performed by one or more processors implemented in a circuit. The one or more processors can be included in a video encoder or in some examples can be included in a video decoder.

[0315] Illustrative examples of the present disclosure include:

[0316] Example 1: A method for encoding video data, the method comprising: using intra-frame sub-partitioning (ISP) to perform division of a coding unit (CU) of video data to form a prediction block set, the prediction block including at least a first prediction block and a second prediction block; grouping multiple prediction blocks from the prediction block set into a first prediction block group (PBG); and reconstructing samples of the prediction blocks included in the first PBG independently of samples of other prediction blocks included in the first PBG.

[0317] Example 2: A method according to Example 1, wherein grouping multiple prediction blocks in a prediction block set into a first PBG includes grouping a first plurality of prediction blocks of the prediction block set into the first PBG, the method further comprising: grouping a second plurality of prediction blocks of the prediction block set into a second PBG, the second plurality of prediction blocks of the prediction block set and the first plurality of prediction blocks of the prediction block set comprising a non-overlapping prediction block set; and reconstructing sample points of the prediction blocks included in the second PBG independently of sample points of other prediction blocks included in the second PBG.

[0318] Example 3: The method according to Example 2, wherein reconstructing the samples of the prediction block included in the second PBG includes reconstructing the samples of the prediction block included in the second PBG based on the samples of the prediction block included in the first PBG.

[0319] Example 4: The method of any of Examples 1 to 3, wherein the first PBG comprises a spot size width of at least four.

[0320] Example 5: A method according to any one of Examples 1 to 4, wherein the size of the first transform block associated with the first PBG is equal to the size of the first prediction block.

[0321] Example 6: The method according to any one of Examples 1 to 5, wherein reconstructing the samples of the prediction block comprises: generating an array of prediction samples having a width equal to a total width of at least the first prediction block and the second prediction block.

[0322] Example 7: The method according to any one of Examples 1 to 6, wherein the first prediction block comprises a sample width less than or equal to two.

[0323] Example 8: The method according to Example 7, wherein the first prediction block and the second prediction block have the same width.

[0324] Example 9: According to the method of any one of Examples 1 to 8, the method further includes: determining the number of prediction block groups based on the width of the coding unit and the width of the first prediction block; coding the CU according to the number of prediction block groups so that the CU includes the number of prediction block groups.

[0325] Example 10: The method of any one of Examples 1 to 9, wherein the method is performed by one or more processors.

[0326] Example 11: According to the method of any one of Examples 1 to 10, the method further includes: encoding one or more syntax elements representing prediction information of the CU of the video data and one or more syntax elements representing residual data of the CU of the video data in the encoded video bitstream.

[0327] Example 12: According to the method of any one of Examples 1 to 11, the method further includes: decoding one or more syntax elements representing prediction information of the CU of the video data and one or more syntax elements representing residual data of the CU of the video data from the encoded video bitstream.

[0328] Example 13: The method of any one of Examples 1 to 12, wherein performing the splitting of the CU comprises performing the splitting of the CU using one or more of vertical splitting, horizontal splitting, or a combination of horizontal and vertical splitting.

[0329] Example 14: A device for encoding video data, the device comprising one or more components for performing the method of any one of Examples 1 to 13. For example, the device of Example 14 may include a memory configured to store video data; and one or more processors, the one or more processors being implemented in a circuit and configured to: perform partitioning of a coding unit (CU) of the video data using intra sub-partitioning (ISP) to form a prediction block set, the prediction block comprising at least a first prediction block and a second prediction block; group a plurality of prediction blocks from the prediction block set into a first prediction block group (PBG); and reconstruct samples of a prediction block included in the first PBG independently of samples of other prediction blocks included in the first PBG.

[0330] Example 15: A device according to Example 14, wherein, in order to group multiple prediction blocks in a prediction block set into a first PBG, one or more processors are configured to group a first plurality of prediction blocks of the prediction block set into the first PBG, and wherein the one or more processors are further configured to: group a second plurality of prediction blocks of the prediction block set into a second PBG, the second plurality of prediction blocks of the prediction block set and the first plurality of prediction blocks of the prediction block set comprising non-overlapping prediction block sets; and reconstruct sample points of the prediction blocks included in the second PBG independently of sample points of other prediction blocks included in the second PBG.

[0331] Example 16: A device according to Example 15, wherein, in order to reconstruct the samples of the prediction block contained in the second PBG, the one or more processors are configured to reconstruct the samples of the prediction block contained in the second PBG based on the samples of the prediction block contained in the first PBG.

[0332] Example 17: The apparatus of any one of Examples 14 to 16, wherein the first PBG comprises a spot size width of at least four.

[0333] Example 18: An apparatus according to any one of Examples 14 to 17, wherein the size of the first transform block associated with the first PBG is equal to the size of the first prediction block.

[0334] Example 19: The apparatus of any one of Examples 14 to 18, wherein, to reconstruct samples of the prediction block, the one or more processors are configured to: generate an array of prediction samples having a width equal to a combined width of at least the first prediction block and the second prediction block.

[0335] Example 20: The apparatus according to any one of Examples 14 to 19, wherein the first prediction block comprises a sample width less than or equal to two.

[0336] Example 21: The apparatus according to Example 20, wherein the first prediction block and the second prediction block have the same width.

[0337] Example 22: A device according to any one of Examples 14 to 21, wherein one or more processors are further configured to: determine the number of prediction block groups based on the width of the coding unit and the width of the first prediction block; and encode the CU according to the number of prediction block groups so that the CU includes the number of prediction block groups.

[0338] Example 23: A device according to any one of Examples 14 to 22, wherein the one or more processors are included in a video encoder.

[0339] Example 24: The apparatus of any one of Examples 14 to 23, wherein the one or more processors are included in a video decoder.

[0340] Example 25: The apparatus of any one of Examples 14 to 24, wherein, to perform the splitting of the CU, the one or more processors are configured to split the CU using vertical splitting.

[0341] In some implementations, the above examples 1-13 and / or 14-25 may be implemented using a computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device to perform some or all of the various operations. For example, a computer-readable storage medium storing instructions may be provided that, when executed, causes one or more processors of a device for decoding and / or encoding video data to: perform partitioning of a coding unit (CU) of video data using intra-frame sub-partitioning (ISP) to form a prediction block set, the prediction block including at least a first prediction block and a second prediction block; group multiple prediction blocks from the prediction block set into a first prediction block group (PBG); and reconstruct samples of the prediction blocks contained in the first PBG independently of samples of other prediction blocks contained in the first PBG.

[0342] In some implementations, the above examples 1-13 and / or 14-25 may be implemented using an apparatus including one or more components for performing some or all of the various operations. For example, an apparatus for encoding video data includes: a component for performing partitioning of a coding unit (CU) of the video data using intra sub-partitioning (ISP) to form a prediction block set, the prediction block including at least a first prediction block and a second prediction block; a component for grouping a plurality of prediction blocks from the prediction block set into a first prediction block group (PBG); and a component for reconstructing samples of a prediction block included in the first PBG independently of samples of other prediction blocks included in the first PBG.

[0343] Example 26: A method for encoding video data, the method comprising: encoding an indication of performance of a generalized prediction of a current encoded block of the video data; dividing the current encoded block into one or more prediction blocks based on the indication; defining one or more prediction block groups (PBGs) based on the one or more prediction blocks; determining an encoding order of the PBGs; and encoding the PBGs in the encoding order.

[0344] Example 27: The method according to Example 26, wherein encoding the PBG comprises: obtaining a predicted value of a sample in the current PBG based on available reference samples of the current PBG and an intra-frame mode for prediction.

[0345] Example 28: A method according to any one of Examples 26 or 27, wherein: dividing the current coded block into one or more prediction blocks includes dividing the current coded block into prediction blocks of size 4×N, allowing only vertical intra-frame prediction mode to be used for prediction of the prediction blocks, and not signaling an indication of the intra-frame prediction mode used for the prediction blocks in a bitstream of an encoded representation of video data containing the prediction blocks.

[0346] Example 29: The method of any one of Examples 26 to 28, wherein encoding comprises decoding.

[0347] Example 30: A method according to any one of Examples 26 to 28, wherein encoding includes encoding.

[0348] Example 31: A device for encoding video data, the device comprising one or more components for performing the method of any one of Examples 26 to 30. For example, the device of Example 31 may include one or more processors, the one or more processors being configured to: encode an indication of performance of a generalized prediction of a current encoded block of video data; divide the current encoded block into one or more prediction blocks based on the indication; define one or more prediction block groups (PBGs) based on the one or more prediction blocks; determine an encoding order of the PBGs; and encode the PBGs in the encoding order.

[0349] Example 32: The apparatus of Example 31, wherein the one or more components include one or more processors implemented in a circuit.

[0350] Example 33: The device according to any of Examples 31 or 32 further includes a memory for storing video data.

[0351] Example 34: The apparatus according to any one of Examples 31 to 33, further comprising a display configured to display the decoded video data.

[0352] Example 35: A device according to any one of Examples 31 to 34, wherein the device comprises one or more of a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0353] Example 36: A device according to any one of Examples 31 to 35, wherein the device comprises a video decoder.

[0354] Example 37: A device according to any one of Examples 31 to 36, wherein the device comprises a video encoder.

[0355] In some implementations, the above examples 26-30 and / or 31-37 may be implemented using a computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device to perform some or all of the various operations. For example, a computer-readable storage medium storing instructions may be provided that, when executed, causes one or more processors of a device for decoding and / or encoding video data to: encode an indication of the performance of a generalized prediction of a current coded block of video data; divide the current coded block into one or more prediction blocks based on the indication; define one or more prediction block groups (PBGs) based on the one or more prediction blocks; determine a coding order for the PBGs; and encode the PBGs in the coding order.

[0356] In some implementations, the above examples 26-30 and / or 31-37 may be implemented using an apparatus including one or more components for performing some or all of the various operations. For example, an apparatus for encoding video data includes: a component for encoding an indication of the performance of a generalized prediction of a current coded block of video data; a component for dividing the current coded block into one or more prediction blocks based on the indication; a component for defining one or more prediction block groups (PBGs) based on the one or more prediction blocks; a component for determining a coding order of the PBGs; and a component for encoding the PBGs in the coding order.

[0357] It should be appreciated that, according to examples, certain actions or events of any technology described herein may be performed in different sequences, may be added, merged, or omitted together (e.g., not all described actions or events are necessary for the practice of the technology). In addition, in some examples, actions or events may be processed concurrently rather than sequentially, for example, by multithreading, interrupt processing, or multiple processors.

[0358] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a non-temporary tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, codes, and / or data structures to implement the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0359] As an example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic memory devices, flash memory or any other medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are sent from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave), the definition of the medium includes coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other temporary media, but are directed to non-temporary tangible storage media. Disks and optical disks as used in this application include compact discs (CDs), laser optical disks, optical optical disks, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein disks generally reproduce data magnetically, while optical disks reproduce data optically with lasers. The above combination should also be included in the scope of computer-readable media.

[0360] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuitry" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described in this application may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated in a combined codec. Similarly, the techniques may be fully implemented in one or more circuits or logic elements.

[0361] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperable hardware units, including one or more processors as described above in combination with appropriate software and / or firmware.

[0362] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for encoding and decoding video data, the method comprising: performing partitioning of a coding unit (CU) of the video data using intra sub-partitioning (ISP) to form a set of prediction blocks, the prediction blocks including at least a first prediction block and a second prediction block; grouping a plurality of prediction blocks from the prediction block set into a first prediction block group (PBG); and The samples of the prediction blocks included in the first PBG are reconstructed independently of the samples of other prediction blocks included in the first PBG.

2. The method according to claim 1, wherein: Grouping the plurality of prediction blocks of the prediction block set into a first PBG comprises grouping a first plurality of prediction blocks of the prediction block set into the first PBG, the method further comprising: grouping a second plurality of prediction blocks of the set of prediction blocks into a second PBG, the second plurality of prediction blocks of the set of prediction blocks and the first plurality of prediction blocks of the set of prediction blocks comprising a non-overlapping set of prediction blocks; and The samples of the prediction block contained in the second PBG are reconstructed independently of the samples of other prediction blocks contained in the second PBG.

3. The method according to claim 2, wherein: Reconstructing the samples of the prediction block included in the second PBG includes reconstructing the samples of the prediction block included in the second PBG based on the samples of the prediction block included in the first PBG.

4. The method according to claim 1, wherein: The first PBG comprises a spot size width of at least four; and / or Herein, a size of a first transform block associated with the first PBG is equal to a size of the first prediction block.

5. The method according to claim 1, wherein: The sample points of the reconstructed prediction block include: A prediction sample array having a width equal to a total width of at least the first prediction block and the second prediction block is generated.

6. The method according to claim 1, wherein: The first prediction block includes a sample width less than or equal to two.

7. The method according to claim 6, wherein: The first prediction block and the second prediction block have the same width.

8. The method according to claim 1, further comprising: Determining the number of prediction block groups based on the width of the coding unit and the width of the first prediction block; The CU is coded according to the number of prediction block groups so that the CU includes the number of prediction block groups.

9. The method according to claim 1, further comprising: encoding, in a coded video bitstream, one or more syntax elements representing prediction information for a CU of video data and one or more syntax elements representing residual data for the CU of video data; or One or more syntax elements representing prediction information for the CU of video data and one or more syntax elements representing residual data for the CU of video data are decoded from the coded video bitstream.

10. A device for encoding and decoding video data, the device comprising: a memory configured to store video data; as well as One or more processors implemented in circuits and configured to perform the method of any one of claims 1-10.

Images