Method, apparatus, and program for encoding and decoding coding unit

Abstract

To provide a method, apparatus, and system for encoding and decoding a block of video samples.SOLUTION: A method of decoding a coding unit from a video bitstream using an entropy decoder comprises: a first decoding step of decoding, from the video bitstream, a luma transform skip flag for a luma component when the coding unit has the luma component; a second decoding step of decoding, from the bitstream, a first chroma transform skip flag for a Cb component and a second chroma transform skip flag for a Cr component when the coding unit has a plurality of chroma components; a determination step of determining whether to decode, from the bitstream, an index for specific transform processing, for the coding unit; and a third decoding step of decoding, from the bitstream, the index for the specific transform processing, for the coding unit, according to a determination result at the determination step.SELECTED DRAWING: Figure 16

Description

[Technical Field]

[0001] REFERENCE TO RELATED APPLICATIONS This application claims the benefit of the filing date under Section 119 of the United States Patent Act of Australia of Australian Patent Application No. 2019275552, filed December 3, 2019, which is incorporated by reference in its entirety as if fully set forth herein.

[0002] The present invention relates generally to digital video signal processing, and more particularly to methods, apparatus and systems for encoding and decoding blocks of video samples. The present invention also relates to a computer program product including a computer-readable medium having recorded thereon a computer program for encoding and decoding blocks of video samples. [Background technology]

[0003] Many applications for video coding currently exist, including applications for the transmission and storage of video data. Many video coding standards have been developed, and others are currently under development. Recent progress in video coding standardization has led to the formation of a group known as the "Joint Video Experts Team" (JVET). The Joint Video Experts Team (JVET) includes members of the International Telecommunication Union (ITU) Telecommunication Standardization Sector (ITU-T) Study Group 16, Research Task 6 (SG16 / Q6), also known as the "Video Coding Experts Group" (VCEG), and members of the International Organization for Standardization / International Electrotechnical Commission Joint Technical Committee 1 / Subcommittee 29 / Working Group 11 (ISO / IEC JTC1 / SC29 / WG11), also known as the "Moving Picture Experts Group" (MPEG).

[0004] The Joint Video Experts Team (JVET) issued a Call for Proposals (CfP) and analyzed responses at its 10th meeting in San Diego, USA. The submitted proposals demonstrated video compression capabilities significantly superior to those of the current state-of-the-art video compression standard, High Efficiency Video Coding (HEVC). Based on these results, it was decided to launch a project to develop a new video compression standard, Versatile Video Coding (VVC). VVC requires ever-higher compression performance due to the increasing demand for high-performance video formats (higher resolutions and frame rates) and for service delivery over wide area networks (WANs), where bandwidth costs are relatively high. Use cases such as immersive video require real-time encoding and decoding of such high-order formats. For example, cube-map projection (CMP) may use 8K formats, even if the "viewport" where the final rendering is performed has a lower resolution. VVC must be implementable on modern silicon processes and offer an acceptable trade-off between achieved performance and implementation cost. Implementation cost can be considered in terms of one or more of the following: silicon area, CPU processor load, memory utilization, and bandwidth. Higher-order video formats can be processed by dividing the frame region into multiple sections and processing each section in parallel. A bitstream constructed from multiple sections of a compressed frame suitable for decoding by a "single-core" decoder (i.e., frame-level constraints including bitrate) is allocated to each section according to the application's needs.

[0005] Video data contains a sequence of multiple frames of image data, each containing one or more color channels. Typically, one primary color channel and two secondary color channels are required. The primary color channel is commonly referred to as the "luma" channel, and the secondary color channel(s) are commonly referred to as the "chroma" channels. Video data is typically displayed in the RGB (red-green-blue) color space, which has a high degree of correlation between the three components. The representation of video data seen by encoders and decoders often uses a color space such as YCbCr. YCbCr aggregates luminance, mapped to "luma" by a transfer function, in the Y (primary) channel and chroma in the Cb and Cr (secondary) channels. Due to the use of uncorrelated YCbCr signals, the statistics of the luma channel differ significantly from those of the chroma channels. The main difference is that after quantization, the chroma channels have relatively fewer significant coefficients for a given block compared to the coefficients of the corresponding luma channel block. Additionally, the Cb and Cr channels may be spatially sampled (subsampled) at a lower rate than the luma channel, for example, half horizontally and half vertically, resulting in a format known as a "4:2:0 chroma format." The 4:2:0 chroma format is used in consumer applications such as Internet video streaming, television broadcasting, and Blu-ray disc storage. Subsampling the Cb and Cr channels at half rate horizontally but not vertically is known as a "4:2:2 chroma format." The 4:2:2 chroma format is often used in professional applications, such as shooting video for film production. Because of its higher sampling rate, the 4:2:2 chroma format produces video that is more resistant to editing processes such as color grading. Material in the 4:2:2 chroma format is often converted to the 4:2:0 chroma format and then encoded for distribution to consumers. In addition to the chroma format, video is also characterized by its resolution and frame rate.Resolutions include 3840x2160 ultra-high definition (UHD) and 7680x4320 "8K," and frame rates include 60Hz and 120Hz. Luma sample rates range from approximately 500 megasamples / second to several gigasamples / second. For 4:2:0 chroma formats, the sample rate for each chroma channel is 1 / 4 the luma sample rate, and for 4:2:2 chroma formats, the sample rate for each chroma channel is 1 / 2 the luma sample rate.

[0006] The VVC standard is a "block-based" codec, where a frame is first divided into an array of square regions known as "coding tree units" (CTUs). If a frame cannot be integer-divided into multiple CTUs, CTUs along the left and bottom edges may be truncated to fit the frame size. CTUs typically occupy relatively large regions, such as 128x128 luma samples. However, CTUs at the right and bottom edges of the frame may have smaller areas. The "coding tree" associated with each CTU may be a single tree for both the luma and chroma channels (a "shared tree"), or it may contain "forks" in separate trees for the luma and chroma channels (a "dual tree"). The coding tree defines the decomposition of the CTU's region into a set of blocks called "coding units" (CUs). CBs are processed for encoding or decoding in a specific order. Separate coding trees for luma and chroma typically start at a granularity of 64x64 luma samples, above which a shared tree exists. Because the 4:2:0 chroma format is used, a separate coding tree structure starts with a 64x64 luma sample granularity, and a chroma coding tree with a 32x32 chroma sample region is placed in the structure. The term "unit" refers to a coding tree that spans all color channels of the block's original coding tree. A single coding tree results in a coding unit with one luma coding block and two chroma coding blocks. The luma branch of another coding tree results in coding units with one luma coding block each, and the chroma branch of another coding tree results in coding units with a pair of chroma blocks each. The CUs mentioned above are also associated with "prediction units" (PUs) and "transform units" (TUs), each of which applies to all color channels of the coding tree from which the CU is derived. Similarly, coding blocks are associated with prediction blocks (PBs) and transform blocks (TBs), each of which applies to a single color channel. A single tree with CUs spanning the color channels of 4:2:0 chroma format video data results in chroma coding blocks with half the width and height of the corresponding luma coding blocks.

[0007] Notwithstanding the above distinction between "unit" and "block," the term "block" can be used as a general term for an area or region of a frame where an operation is applied to all color channels.

[0008] For each CU, a prediction unit (PU) is generated for the contents (sample values) of the corresponding area of ​​the frame data. Furthermore, a representation of the difference between the predicted values ​​and the contents of the region as seen at the input to the encoder (or "spatial domain" residual) is formed. The differences for each color channel are transformed and coded as a sequence of residual coefficients, forming one or more TUs for a given CU. The applied transform may be a discrete cosine transform (DCT) or other transform applied to each block of residual values. This transform is applied in isolation, with two-dimensional transforms performed in two passes. First, a one-dimensional transform is applied to each row of samples in the block to transform the block. Second, a one-dimensional transform is applied to each column of the partial results to transform the partial results, producing a final block of transform coefficients that substantially correlates the residual samples. The VVC standard supports transforms of various sizes, including rectangular blocks with sides that are a power of two. The transform coefficients are quantized for entropy coding into the bitstream. Further non-separable transform stages may be applied. Finally, the application of the transform may be bypassed.

[0009] VVC features intra-frame and inter-frame prediction. Intra-frame prediction uses previously processed samples within a frame to generate a prediction of a current block of samples within a frame. Inter-frame prediction involves using a block of samples from a previously decoded frame to generate a prediction of a current block of samples within a frame. The block of samples from the previously decoded frame is offset from the spatial location of the current block according to a motion vector, often with filtering applied. An intra-frame predicted block can be (i) a uniform sample value ("DC intra-prediction"), (ii) a plane with an offset and horizontal and vertical gradients ("planar intra-prediction"), (iii) a collection of neighboring samples and blocks applied in a specific direction ("angular intra-prediction"), or (iv) the result of a matrix multiplication of neighboring samples with selected matrix coefficients. Further discrepancies between the predicted block and the corresponding input samples can be corrected to some extent by encoding a "residual" into the bitstream. The residual is typically transformed from the spatial domain to the frequency domain (in a "primary transform" domain) to form residual coefficients, and may be further transformed by application of a "secondary transform" (to produce residual coefficients in a "secondary transform domain"). The residual coefficients are quantized according to a quantization parameter, resulting in a reduced bitstream bitrate, at the expense of a loss of accuracy in the reconstruction of the samples produced at the decoder.

[0010] The quantization parameter may vary between frames and within each frame. Varying the quantization parameter within a frame is a typical example of a "rate-controlled" encoder. A rate-controlled encoder attempts to generate a bitstream at a substantially constant bitrate, regardless of the statistics of the received input samples, such as noise characteristics or the degree of motion. Because bitstreams are typically transmitted over networks with limited bandwidth, rate control is a widely used technique to ensure reliable performance over the network, regardless of variations in the original frames input to the encoder. When frames are coded in parallel sections, the use of rate control requires flexibility because different sections require different fidelity.

[0011] Additionally, implementation costs such as memory usage, accuracy, and communication efficiency are also important. Summary of the Invention

[0012] SUMMARY OF THE INVENTION It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing devices.

[0013] One aspect of the present invention is a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream, the coding unit having one luma color channel and at least one chroma color channel, the method comprising: decoding a luma transform skip flag from the video bitstream for a luma transform block of the coding unit; decoding at least one chroma transform skip flag from the video bitstream, each decoded chroma transform skip flag corresponding to one of the at least one chroma transform blocks of the coding unit; determining a secondary transformation index, the determining comprising: decoding a secondary transform index from the video bitstream when at least one of the luma transform skip flag and the at least one chroma transform skip flag indicates that a transform of a respective transform block is not skipped; and determining the secondary transform index to indicate that a secondary transform is not applied when both the luma transform skip flag and the at least one chroma transform skip flag indicate that a transform of the respective transform block is skipped; said determining including: transforming the luma transform block and the at least one chroma transform block according to the decoded luma transform skip flag, the at least one chroma transform skip flag, and the determined secondary transform index to decode the coding unit; The present invention provides a method comprising:

[0014] According to another aspect, the decoded luma transform skip flag has a different value than the at least one chroma transform skip flag.

[0015] According to another aspect, when the decoded luma transform skip flag indicates that the transform of a luma block is skipped, the secondary transform index is decoded for the at least one chroma transform block based on the decoded at least one chroma skip flag.

[0016] According to another aspect, the transforming includes one of skipping application of a secondary transform or selecting one of two secondary transform kernels for application based on the determined secondary transform index.

[0017] Another aspect of the present invention is a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream, the coding unit having at least one chroma color channel, the method comprising: decoding at least one chroma transform skip flag from the video bitstream, each chroma transform skip flag corresponding to one of the at least one chroma transform blocks of the coding unit; determining a secondary transform index for the at least one chroma transform block of the coding unit, the determining including: decoding the secondary transform index from the video bitstream when any of the at least one chroma transform skip flag indicates that a transform is applied to a respective chroma transform block; and determining the secondary transform index to indicate that a secondary transform is not applied when all of the chroma transform skip flags indicate that a transform of the respective transform block is skipped; said determining including: transforming each of the at least one chroma transform block according to a respective chroma transform skip flag and the determined secondary transform index to decode the coding unit; The present invention provides a method comprising:

[0018] Another aspect of the present disclosure is a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream, the coding unit having one luma color channel and at least one chroma color channel, the method comprising: decoding a luma transform skip flag from the video bitstream for a luma transform block of the coding unit; decoding at least one chroma transform skip flag from the video bitstream, each decoded chroma transform skip flag corresponding to one of the at least one chroma transform blocks of the coding unit; determining a secondary transformation index, the determining comprising: determining the secondary transform index to indicate that a secondary transform is not applied when both the luma transform skip flag and the at least one chroma transform skip flag indicate that a transform of the respective transform block is skipped; decoding a secondary transform index from the video bitstream when the luma transform skip flag and the at least one chroma transform skip flag all indicate that a transform of a respective transform block is not skipped; and said determining including: transforming the luma transform block and the at least one chroma transform block according to the decoded luma transform skip flag, the at least one chroma transform skip flag, and the determined secondary transform index to decode the coding unit; The present invention provides a method comprising:

[0019] Another aspect of the present invention is a non-transitory computer-readable medium having stored thereon a computer program for implementing a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream, the coding unit having one luma color channel and at least one chroma color channel, the method comprising: decoding a luma transform skip flag from the video bitstream for a luma transform block of the coding unit; decoding at least one chroma transform skip flag from the video bitstream, each decoded chroma transform skip flag corresponding to one of the at least one chroma transform blocks of the coding unit; determining a secondary transformation index, the determining comprising: decoding a secondary transform index from the video bitstream when at least one of the luma transform skip flag and the at least one chroma transform skip flag indicates that a transform of a respective transform block is not skipped; and determining the secondary transform index to indicate that a secondary transform is not applied when both the luma transform skip flag and the at least one chroma transform skip flag indicate that a transform of the respective transform block is skipped; said determining including: transforming the luma transform block and the at least one chroma transform block according to the decoded luma transform skip flag, the at least one chroma transform skip flag, and the determined secondary transform index to decode the coding unit; A non-transitory computer-readable medium is provided, comprising:

[0020] Another aspect of the present invention is a system comprising: Memory and a processor configured to execute code stored in the memory to implement a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream, the coding unit having at least one chroma color channel; and wherein the method comprises: decoding at least one chroma transform skip flag from the video bitstream, each chroma transform skip flag corresponding to one of the at least one chroma transform blocks of the coding unit; determining a secondary transform index for the at least one chroma transform block of the coding unit, the determining including: decoding the secondary transform index from the video bitstream when any of the at least one chroma transform skip flag indicates that a transform is applied to a respective chroma transform block; and determining the secondary transform index to indicate that a secondary transform is not applied when all of the at least one chroma transform skip flags indicate that a transform of the respective transform block is skipped; and said determining including: transforming each of the at least one chroma transform block according to a respective chroma transform skip flag and the determined secondary transform index to decode the coding unit; The present invention provides a system including:

[0021] Another aspect of the present invention is a video decoder comprising: receiving image frames from a video bitstream; determining a coding unit of a coding tree from the coding tree units of the image frame, the coding unit having one luma color channel and at least one chroma color channel; decoding a luma transform skip flag from the video bitstream for a luma transform block of the coding unit; decoding at least one chroma transform skip flag from the video bitstream, each decoded chroma transform skip flag corresponding to one of the at least one chroma transform blocks of the coding unit; determining a secondary transformation index, said determination comprising: decoding a secondary transform index from the video bitstream when at least one of the luma transform skip flag and the at least one chroma transform skip flag indicates that a transform of a respective transform block is not skipped; and determining the secondary transform index to indicate that a secondary transform is not applied when both the luma transform skip flag and the at least one chroma transform skip flag indicate that a transform of the respective transform block is skipped; Including, transforming the luma transform block and the at least one chroma transform block according to the decoded luma transform skip flag, the at least one chroma transform skip flag, and the determined secondary transform index to decode the coding unit; The present invention provides a video decoder configured to:

[0022] Another aspect of the present invention is a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream, the method comprising: determining a scan pattern for a transform block of the coding unit, wherein the scan pattern traverses the transform block by progressing through multiple non-overlapping collections of sub-blocks of residual coefficients, and the scan pattern progresses from the current collection to a next collection of the multiple collections after completing a scan of the current collection; decoding residual coefficients from the video bitstream according to the determined scan pattern; determining a plurality of transform selection indexes for the coding unit, the determining including: decoding the multiple transform selection indexes from the video bitstream when a last significant coefficient encountered along the scan pattern is at or within a threshold orthogonal position of the transform block; determining the multiple transform selection index to indicate that the multiple transform selection is not being used when a position of the last significant residual coefficient of the transform block along the scan pattern is outside the threshold orthogonal position; said determining including: transforming the decoded residual coefficients applying a transform according to the multiple transform selection indexes to decode the coding unit; The present invention provides a method comprising:

[0023] Another aspect of the present invention is a non-transitory computer-readable medium having stored thereon a computer program for implementing a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream, the method comprising: determining a scan pattern for a transform block of the coding unit, wherein the scan pattern traverses the transform block by progressing through multiple non-overlapping collections of sub-blocks of residual coefficients, and the scan pattern progresses from the current collection to a next collection of the multiple collections after completing a scan of the current collection; decoding residual coefficients from the video bitstream according to the determined scan pattern; determining a plurality of transform selection indexes for the coding unit, the determining including: decoding the multiple transform selection indexes from the video bitstream when a last significant coefficient encountered along the scan pattern is at or within a threshold orthogonal position of the transform block; determining the multiple transform selection index to indicate that the multiple transform selection is not being used when a position of the last significant residual coefficient of the transform block along the scan pattern is outside the threshold orthogonal position; said determining including: transforming the decoded residual coefficients applying a transform according to the multiple transform selection indexes to decode the coding unit; A non-transitory computer-readable medium is provided, comprising:

[0024] Another aspect of the present invention is a system comprising: Memory and a processor configured to execute code stored in the memory to implement a method for decoding a coding unit of a coding tree from a coding tree unit of an image frame from a video bitstream; wherein the method comprises: determining a scan pattern for a transform block of the coding unit, wherein the scan pattern traverses the transform block by progressing through multiple non-overlapping collections of sub-blocks of residual coefficients, and the scan pattern progresses from the current collection to a next collection of the multiple collections after completing a scan of the current collection; decoding residual coefficients from the video bitstream according to the determined scan pattern; determining a plurality of transform selection indexes for the coding unit, the determining including: decoding the multiple transform selection indexes from the video bitstream when a last significant coefficient encountered along the scan pattern is at or within a threshold orthogonal position of the transform block; determining the multiple transform selection index to indicate that the multiple transform selection is not being used when a position of the last significant residual coefficient of the transform block along the scan pattern is outside the threshold orthogonal position; Including, transforming the decoded residual coefficients by applying a transform according to the multiple transform selection indexes to decode the coding unit; The present invention provides a video decoder configured to:

[0025] Another aspect of the present invention is a video decoder comprising: receiving image frames from a video bitstream; determining a coding unit of a coding tree from the coding tree units of the image frame; determining a scan pattern for a transform block of the coding unit, wherein the scan pattern traverses the transform block by progressing through multiple non-overlapping collections of sub-blocks of residual coefficients, and the scan pattern progresses from the current collection to a next collection of the multiple collections after completing a scan of the current collection; decoding residual coefficients from the video bitstream according to the determined scan pattern; determining a multiple transform selection index for the coding unit, the determining comprising: decoding the multiple transform selection indexes from the video bitstream when a last significant coefficient encountered along the scan pattern is at or within a threshold orthogonal position of the transform block; determining the multiple transform selection index to indicate that the multiple transform selection is not being used when a position of the last significant residual coefficient of the transform block along the scan pattern is outside the threshold orthogonal position; said determining including: transforming the decoded residual coefficients applying a transform according to the multiple transform selection indexes to decode the coding unit; The present invention provides a method comprising:

[0026] Other aspects are also disclosed. [Brief explanation of the drawings]

[0027] At least one embodiment of the present invention will now be described with reference to the following drawings and appendices.

[0028] [Figure 1] FIG. 1 is a schematic block diagram illustrating a video encoding and decoding system.

[0029] [Figure 2A]FIG. 2A is a schematic block diagram of a general-purpose computer system on which one or both of the video encoding and decoding systems of FIG. 1 may be implemented. [Figure 2B] FIG. 2B is a schematic block diagram of a general-purpose computer system on which one or both of the video encoding and decoding systems of FIG. 1 may be implemented.

[0030] [Figure 3] FIG. 3 is a schematic block diagram illustrating the functional modules of a video encoder.

[0031] [Figure 4] FIG. 4 is a schematic block diagram illustrating the functional modules of a video decoder.

[0032] [Figure 5] FIG. 5 is a schematic block diagram illustrating possible divisions of a block into one or more blocks in a versatile video coding tree structure.

[0033] [Figure 6] FIG. 6 is a schematic diagram illustrating the data flow for realizing the permitted division of a block into one or more blocks in a versatile video coding tree structure.

[0034] [Figure 7A] FIG. 7A is a diagram showing an example of dividing a coding tree unit (CTU) into multiple coding units (CUs). [Figure 7B] FIG. 7B is a diagram showing an example of dividing a coding tree unit (CTU) into multiple coding units (CUs).

[0035] [Figure 8A] FIG. 8A illustrates non-separable forward and inverse quadratic transforms performed according to transform blocks of different sizes. [Figure 8B] FIG. 8B illustrates non-separable forward and inverse quadratic transforms performed according to transform blocks of different sizes. [Figure 8C] FIG. 8C illustrates non-separable forward and inverse quadratic transforms performed according to transform blocks of different sizes. [Figure 8D] FIG. 8D illustrates non-separable forward and inverse quadratic transforms performed according to transform blocks of different sizes.

[0036] [Figure 9] FIG. 9 is a diagram showing a set of application regions of the secondary transform for transform blocks of various sizes.

[0037] [Figure 10] FIG. 10 shows the syntax structure of a bitstream with multiple slices, each containing multiple coding units.

[0038] [Figure 11] FIG. 11 is a diagram illustrating a syntax structure of a bitstream having a shared tree of luma coding units and chroma coding units of coding tree units.

[0039] [Figure 12] FIG. 12 is a diagram showing a syntax structure of a bitstream in which the luma coding unit and the chroma coding unit of the coding tree unit are in separate trees.

[0040] [Figure 13] FIG. 13 illustrates how a frame is encoded as a sequence of coding units into a bitstream containing one or more slices.

[0041] [Figure 14] FIG. 14 is a diagram illustrating a method for encoding a coding unit into a bitstream.

[0042] [Figure 15]FIG. 15 illustrates how a frame is decoded from a bitstream as a sequence of coding units arranged into slices.

[0043] [Figure 16] FIG. 16 is a diagram illustrating a method for decoding a coding unit from a bitstream.

[0044] [Figure 17] FIG. 17 is a diagram showing a conventional scan pattern for 32×32 TB.

[0045] [Figure 18] FIG. 18 shows an example of a 32×32TB scan pattern for use with the described device.

[0046] [Figure 19] FIG. 19 shows a diagram of a TB of size 8×32 divided into collections for the described device.

[0047] [Figure 20] FIG. 20 shows different example scan patterns for 32×32TB used in the described device. DETAILED DESCRIPTION OF THE INVENTION

[0048] Where steps and / or features having the same reference numerals are referenced in any one or more of the accompanying drawings, those steps and / or features have the same function(s) or operation(s) for the purposes of this specification, unless a contrary intention appears.

[0049] The syntax of a video compression standard's bitstream format is defined as a hierarchy of "syntax structures." Each syntax structure defines a set of syntax elements, some of which may be conditioned on others. Compression efficiency is improved when the syntax allows only combinations of syntax elements that correspond to useful combinations of tools. Complexity is further reduced by prohibiting combinations of syntax elements that, while possible to implement, provide insufficient compression benefit relative to the resulting implementation cost.

[0050] 1 is a schematic block diagram illustrating the functional modules of a video encoding and decoding system 100. The system 100 signals primary and secondary transform parameters such that compression efficiency gains are achieved.

[0051] System 100 includes source device 110 and destination device 130. Communication channel 120 is used to communicate encoded video information from source device 110 to destination device 130. In some arrangements, either or both of source device 110 and destination device 130 may comprise respective mobile phone handsets or “smartphones,” in which case communication channel 120 is a wireless channel. In other configurations, source device 110 and destination device 130 may comprise video conferencing equipment, in which case communication channel 120 is typically a wired channel, such as an Internet connection. Furthermore, source device 110 and destination device 130 may comprise any of a wide range of devices, including devices supporting over-the-air television broadcasts, cable television applications, Internet video applications (including streaming), and applications in which encoded video data is ingested onto some computer-readable storage medium, such as a hard disk drive in a file server.

[0052] As shown in FIG. 1, source device 110 includes video source 112, video encoder 114, and transmitter 116. Video source 112 typically constitutes a source of captured video frame data (shown as 113), such as an image capture sensor, a previously captured video sequence stored on a non-transitory storage medium, or a video feed from a remote image capture sensor. Video source 112 may also be the output of a computer graphics card, e.g., displaying the video output of an operating system or various applications running on a computing device such as a tablet computer. Examples of source device 110 that may include an image capture sensor as video source 112 include smartphones, video camcorders, professional video cameras, and network video cameras.

[0053] Video encoder 114 converts (or "encodes") captured frame data (indicated by arrow 113) from video source 112 into a bitstream (indicated by arrow 115), as further described with reference to FIG. 3. Bitstream 115 is transmitted by transmitter 116 as coded video data (or "coded video information") over communication channel 120. Bitstream 115 may also be stored in non-transitory storage device 122, such as "flash" memory or a hard disk drive, until later transmitted over communication channel 120, or in lieu of transmission over communication channel 120. For example, coded video data may be provided on demand to a customer over a wide area network (WAN) for video streaming applications.

[0054] Destination device 130 includes receiver 132, video decoder 134, and display device 136. Receiver 132 receives encoded video data from communication channel 120 and passes the received video data as a bitstream (indicated by arrow 133) to video decoder 134. Video decoder 134 then outputs decoded frame data (indicated by arrow 135) to display device 136. Decoded frame data 135 has the same chroma format as frame data 113. Examples of display device 136 include a cathode ray tube, a smartphone or tablet computer, a computer monitor, or an LCD display such as those found in standalone televisions. The functionality of source device 110 and destination device 130 may also be embodied in a single device, such as a mobile phone or tablet computer. The decoded frame data may be further transformed before presentation to a user. For example, a "viewport" with a particular latitude and longitude may provide a 360° view of a scene. o To represent a view, it can be rendered from the decoded frame data using a projection format.

[0055] Notwithstanding the exemplary apparatus described above, each of source device 110 and destination device 130 may be configured within a general-purpose computing system, typically with a combination of hardware and software components. FIG. 2A illustrates such a computer system 200, which includes a computer module 201, input devices such as a keyboard 202, a mouse pointer device 203, a scanner 226, a camera 227 that may be configured as a video source 112, and a microphone 280, and output devices such as a printer 215, a display device 214 that may be configured as a display device 136, and a loudspeaker 217. An external modulator-demodulator (modem) transceiver device 216 may be used by computer module 201 to communicate with a communications network 220 via connection 221. Communications network 220, which may represent communications channel 120, may be a WAN such as the Internet, a cellular communications network, or a private WAN. If connection 221 is a telephone line, modem 216 may be a conventional “dial-up” modem. Alternatively, the modem 216 may be a broadband modem if the connection 221 is a high-capacity (e.g., cable or optical) connection, or a wireless modem may be used for a wireless connection to the communications network 220. The transceiver device 216 may provide the functionality of the transmitter 116 and receiver 132, and the communications channel 120 may be embodied in the connection 221.

[0056] The computer module 201 typically includes at least one processor unit 205 and a memory unit 206. For example, the memory unit 206 may include semiconductor random access memory (RAM) and semiconductor read-only memory (ROM). The computer module 201 also includes a number of input / output (I / O) interfaces, including an audio-video interface 207 that couples to a video display 214, a loudspeaker 217, and a microphone 280; an I / O interface 213 that couples to a keyboard 202, a mouse 203, a scanner 226, a camera 227, and optionally a joystick or other human interface device (not shown); and an interface 208 for an external modem 216 and a printer 215. The signal from the audio-video interface 207 to the computer monitor 214 is typically the output of a computer graphics card. In some implementations, the modem 216 may be integrated into the computer module 201, for example, in the interface 208. The computer module 201 also has a local network interface 211, which enables coupling of the computer system 200 via a connection 223 to a local area communications network 222, known as a local area network (LAN). As shown in FIG. 2A, the local communications network 222 may also be coupled to a wide area network 220 via a connection 224, which would typically include a so-called "firewall" device or a device with similar functionality. The local network interface 211 may consist of an Ethernet™ circuit card, a Bluetooth™ wireless device, or an IEEE 802.11 wireless device, although many other types of interfaces may be implemented for the interface 211. The local network interface 211 may also provide the functionality of the transmitter 116 and receiver 132, and the communications channel 120 may also be embodied in the local communications network 222.

[0057] The I / O interfaces 208 and 213 may provide either or both serial and parallel connections, the former typically implemented according to the Universal Serial Bus (USB) standard and having a corresponding USB connector (not shown). The storage device 209 typically includes a hard disk drive (HDD) 210. Other storage devices, such as floppy disk drives and magnetic tape drives (not shown), may also be used. An optical disk drive 212 is typically provided to serve as a non-volatile source of data. Portable memory devices, such as optical disks (e.g., CD-ROMs, DVDs, Blu-ray Discs™), USB-RAM, portable, external hard drives, and floppy disks, may also be used as suitable data sources for the computer system 200. Typically, any of the HDD 210, optical drive 212, and networks 220 and 222 may also be configured to operate as a video source 112 or as a destination for decoded video data to be stored for playback via the display 214. The source device 110 and the destination device 130 of the system 100 may be embodied in a computer system 200 .

[0058] The components 205-213 of the computer module 201 typically communicate via an interconnected bus 204 and in a manner which results in a conventional mode of operation of the computer system 200 known to those skilled in the art. For example, the processor 205 is coupled to the system bus 204 using a connection 218. Similarly, the memory 206 and optical disk drive 212 are coupled to the system bus 204 by a connection 219. Examples of computers on which the described apparatus can be implemented include IBM-PCs and compatibles, Sun SPARC stations, Apple Mac™ or a similar computer systems.

[0059] Where appropriate or desired, video encoder 114 and video decoder 134, as well as the methods described below, may be implemented using computer system 200. In particular, video encoder 114, video decoder 134, and the described methods may be implemented as one or more software application programs 233 executable within computer system 200. In particular, video encoder 114, video decoder 134, and the steps of the described methods are effected by instructions 231 (see FIG. 2B ) in software 233 executed within computer system 200. The software instructions 231 may be formed as one or more code modules, each for performing one or more specific tasks. The software may also be divided into two separate portions, where a first portion and corresponding code modules perform the described methods and a second portion and corresponding code modules manage a user interface between the first portion and a user.

[0060] The software may be stored on a computer-readable medium, including, for example, the storage devices described below. The software is loaded from the computer-readable medium into computer system 200 and then executed by computer system 200. A computer-readable medium having such software or a computer program recorded thereon is a computer program product. Use of the computer program product in computer system 200 preferably results in an advantageous apparatus for implementing video encoder 114, video decoder 134, and the methods described.

[0061] The software 233 is typically stored on the HDD 210 or the memory 206. The software is loaded into the computer system 200 from a computer-readable medium and executed by the computer system 200. Thus, for example, the software 233 may be stored on an optically readable disk storage medium (e.g., a CD-ROM) 225 that is read by the optical disk drive 212.

[0062] In some examples, the application program 233 may be encoded on one or more CD-ROMs 225 and supplied to the user, and read via the corresponding drive 212, or alternatively, read by the user from the network 220 or 222. Furthermore, the software may also be loaded into the computer system 200 from other computer-readable media. A computer-readable storage medium refers to any non-transitory, tangible storage medium that provides recorded instructions and / or data to the computer system 200 for execution and / or processing. Examples of such storage media include floppy disks, magnetic tapes, CD-ROMs, DVDs, Blu-ray Discs™, hard disk drives, ROMs or integrated circuits, USB memory, magneto-optical disks, or computer-readable cards such as PCMCIA cards, whether such devices are internal or external to the computer module 201. Examples of transitory or non-tangible computer-readable transmission media that may also participate in providing software, application programs, instructions and / or video data or encoded video data to the computer module 401 include wireless or infrared transmission paths, as well as network connections to other computers or network devices, and the Internet or intranet, including email transmissions, information stored on websites, etc.

[0063] The second portion of application program 233 and corresponding code modules described above may be executed to implement one or more graphical user interfaces (GUIs) that are rendered or otherwise represented on display 214. A user of computer system 200 and applications, typically through manipulation of keyboard 202 and mouse 203, can manipulate the interface in a functionally adaptable manner to provide control commands and / or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as audio interfaces utilizing voice prompts output via loudspeaker 217 or user voice commands input via microphone 280.

[0064] Figure 2B is a detailed schematic block diagram of processor 205 and "memory" 234. Memory 234 represents the logical aggregation of all memory modules (including HDD 209 and semiconductor memory 206) accessible by computer module 201 of Figure 2A.

[0065] When the computer module 201 is initially powered on, a power-on self-test (POST) program 250 is executed. The POST program 250 is typically stored in the ROM 249 of the semiconductor memory 206 of FIG. 2A. Note that hardware devices that store software, such as the ROM 249, are sometimes called firmware. The POST program 250 inspects the hardware within the computer module 201 to ensure proper functionality and typically checks the processor 205, the memory 234 (209, 206), and a basic input / output system software (BIOS) module 251, which is also typically stored in the ROM 249. If the POST program 250 executes successfully, the BIOS 251 activates the hard disk drive 210 of FIG. 2A. The activation of the hard disk drive 210 causes a bootstrap loader program 252 resident on the hard disk drive 210 to be executed via the processor 205. This loads the operating system 253 into the RAM memory 206, thereby initiating operation of the operating system 253. Operating system 253 is a system-level application executable by processor 205 that performs a variety of high-level functions, including processor management, memory management, device management, storage management, software application interface, and general-purpose user interface.

[0066] The operating system 253 manages the memory 234 (209, 206) so that each process or application running on the computer module 201 has enough memory to execute without conflicting with memory allocated to another process. Furthermore, each process must appropriately use the different types of memory available in the computer system 200 of FIG. 2A so that it can execute effectively. Thus, the aggregate memory 234 is not intended to describe how specific segments of memory are allocated (unless otherwise noted), but rather to provide a general view of the memory accessible by the computer system 200 and how such is used.

[0067] As shown in FIG. 2B, processor 205 includes a number of functional modules, including a control unit 239, an arithmetic logic unit (ALU) 240, and a local or internal memory 248, sometimes referred to as a cache memory. Cache memory 248 typically includes a number of storage registers 244-246 in a register section. One or more internal buses 241 functionally interconnect these functional modules. Processor 205 also typically has one or more interfaces 242 for communicating with external devices via system bus 204 using connection 218. Memory 234 is coupled to bus 204 using connection 219.

[0068] Application program 233 includes a set of instructions 231, which may include conditional branch instructions and loop instructions. Program 233 may also include data 232 used in the execution of program 233. Instructions 231 and data 232 are stored in memory locations 228, 229, 230 and 235, 236, 237, respectively. Depending on the relative sizes of instruction 231 and memory locations 228-230, a particular instruction may be stored in a single memory location, as depicted by the instruction shown in memory location 230. Alternatively, the instruction may be segmented into several parts, each stored in a separate memory location, as depicted by the instruction segments shown in memory locations 228 and 229.

[0069] Generally, the processor 205 is given a set of instructions to execute. The processor 205 waits for subsequent input, to which the processor 205 responds by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 202, 203, data received from an external source via one of the networks 220, 202, data obtained from one of the storage devices 206, 209, or data obtained from a storage medium 225 inserted into a corresponding reader 212, all of which are depicted in FIG. 2A. Execution of the set of instructions may in some cases involve the output of data. Execution may also include storing data or variables in memory 234.

[0070] The video encoder 114, video decoder 134, and described methods may use input variables 254 that are stored in memory 234 at corresponding memory locations 255, 256, 257. The video encoder 114, video decoder 134, and described methods generate output variables 261 that are stored in memory 234 at corresponding memory locations 262, 263, 264. Intermediate variables 258 may be stored in memory locations 259, 260, 266, 267.

[0071] 2B, registers 244, 245, 246, arithmetic logic unit (ALU) 240, and control unit 239 cooperate to perform the sequence of micro-operations required to perform a "fetch, decode, and execute" cycle for every instruction in the instruction set that makes up program 233. Each fetch, decode, and execute cycle includes: a fetch operation to fetch or read an instruction 231 from a memory location 228, 229, 230; a decoding operation in which the control unit 239 determines which instructions have been fetched; and An execution operation in which the control unit 239 and / or ALU 240 executes the instruction.

[0072] Thereafter, a further fetch, decode, and execute cycle of the next instruction may be performed. Similarly, a store cycle may be performed in which control unit 239 stores or writes a value to memory location 232.

[0073] Each step or sub-process in the methods of Figures 13-16 described below is associated with one or more segments of program 233 and is typically performed by register units 244, 245, 247, ALU 240, and control unit 239 within processor 205 working together to perform fetch, decode, and execute cycles for all instructions in the instruction set for the indicated segment of program 233.

[0074] Figure 3 is a schematic block diagram illustrating the functional modules of video encoder 114. Figure 4 is a schematic block diagram illustrating the functional modules of video decoder 134. Generally, data passes between the functional modules in video encoder 114 and the functional modules in video decoder 134 in groups of samples or coefficients, such as blocks divided into fixed-size sub-blocks, or as arrays. Video encoder 114 and video decoder 134 may be implemented using a general-purpose computer system 200, as shown in Figures 2A and 2B, where the various functional modules may be implemented by dedicated hardware within computer system 200, by software executable within computer system 200, such as one or more software code modules of a software application program 233 resident on hard disk drive 205, and whose execution is controlled by processor 205. Alternatively, video encoder 114 and video decoder 134 may be implemented by a combination of dedicated hardware and software executable within computer system 200. The video encoder 114, the video decoder 134, and the described methods may alternatively be implemented with dedicated hardware, such as one or more integrated circuits, that perform functions or sub-functions of the described methods. Such dedicated hardware may include a graphics processing unit (GPU), a digital signal processor (DSP), an application specific standard product (ASSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or one or more microprocessors and associated memory. In particular, the video encoder 114 includes modules 310-386, and the video decoder 134 includes modules 420-496, each of which may be implemented as one or more software code modules of the software application program 233.

[0075] The video encoder 114 of FIG. 3 is an example of a Versatile Video Coding (VVC) video encoding pipeline, although other video codecs may be used to perform the processing stages described herein. The video encoder 114 receives captured frame data 113, such as a series of frames, each frame including one or more color channels. The frame data 113 includes a two-dimensional array of luma ("luma channel") and chroma ("chroma channel") samples arranged in a "chroma format," such as a 4:0:0, 4:2:0, 4:2:2, or 4:4:4 chroma format. The block partitioner 310 first divides the frame data 113 into CTUs, which are generally square in shape and configured to use a specific size for the CTU. The size of the CTU may be, for example, 64x64, 128x128, or 256x256 luma samples.

[0076] The block partitioner 310 further divides each CTU into one or more CUs according to either the shared coding tree or the luma and chroma coding trees at the point where the shared coding tree branches into a luma branch and a chroma branch. The luma channel may also be referred to as a primary color channel. Each chroma channel may also be referred to as a secondary color channel. CUs may have various sizes and include both square and non-square aspect ratios. The operation of the block partitioner 310 is further described with reference to Figures 13 and 14. However, in the VVC standard, CUs / CBs, PUs / PBs, and TUs / TBs always have side lengths that are powers of two. Thus, a current CU, denoted as 312, proceeds with an iteration for one or more blocks of the CTU according to the CTU's shared coding tree or the luma and chroma coding trees and is output from the block partitioner 310. Options for partitioning CTUs into CBs are further described below with reference to Figures 5 and 6.

[0077] The CTUs resulting from the first division of frame data 113 may be scanned in raster scan order and grouped into one or more "slices." A slice may be an "intra" (or "I") slice. An intra slice (I slice) does not contain inter-predicted CUs, e.g., only intra prediction is used. Alternatively, a slice may be unidirectionally predictive or bidirectionally predictive ("P" or "B" slice, respectively), indicating the additional availability of one or two reference blocks for predicting a CU, known as "unidirectional prediction" and "bidirectional prediction," respectively.

[0078] In an I slice, the coding tree of each CTU may branch at the 64x64 level or lower into two coding trees, one for luma and one for chroma. The use of separate trees allows for different block structures for luma and chroma within the luma 64x64 region of a CTU. For example, a large chroma CB can be populated with many small luma CBs, or vice versa. In a P or B slice, a single coding tree for a CTU defines a common block structure for luma and chroma. The resulting blocks of the single tree can be intra-predicted or inter-predicted.

[0079] For each CTU, the video encoder 114 operates in two stages. In the first stage (called the "search" stage), the block partitioner 310 tests various potential configurations of the coding tree. Each potential configuration of the coding tree has an associated "candidate" CU. In the first stage, various candidate CUs are tested to select a CU that provides relatively high compression efficiency with relatively low distortion. This testing typically involves Lagrangian optimization, whereby candidate CUs are evaluated based on a weighted combination of rate (coding cost) and distortion (error relative to the input frame data 113). The "best" candidate CU (the CU with the lowest evaluated rate / distortion) is selected for subsequent encoding into the bitstream 115. The evaluation of candidate CUs includes the options of using the CU for a given region, further dividing the region according to various partitioning options and encoding each of the resulting smaller regions with additional CUs, or further dividing the region. As a result, both the coding tree and the CU itself are selected in the search stage.

[0080] The video encoder 114 generates a prediction block (PU), indicated by arrow 320, for each CU, e.g., CU 312. PU 320 is a prediction of the content of the associated CU 312. Subtractor module 322 generates a difference, indicated as 324 (or "residual," indicating that the difference is in the spatial domain), between PB 320 and CB 312. Difference 324 is a block-sized difference between corresponding samples of PU 320 and CU 312. Difference 324 is a block-sized array of differences between corresponding samples of PU 320 and CU 312, generated for each color channel of CU 312. If a primary transform and (optionally) a secondary transform are performed, difference 324 is transformed in modules 326 and 330 and passed via multiplexing 333 to quantization module 334 for quantization. If the transform is skipped, difference 324 is passed via multiplexing 333 directly to quantization module 334 for quantization. The choice between transform and transform skip is made independently for each TB associated with the CU 312. The resulting quantized residual coefficients are represented as TBs (for each color channel of the CU 312) indicated by arrows 336. The PU 320 and associated TBs 336 are typically selected from one of many possible candidate CUs, for example, based on estimated cost or distortion.

[0081] A candidate CU is a CU resulting from one of the prediction modes available to the video encoder 114 for the associated PB and the resulting residual. When combined with the predicted PB in the video decoder 114, the addition of the TB 336 after the inverse transform to the spatial domain reduces the difference between the decoded CU and the original CU 312, at the expense of additional signaling in the bitstream.

[0082] Each candidate coding block (CU), i.e., a prediction block (PU) combined with one transform block (TB), therefore has an associated coding cost (or “rate”) and an associated differential (or “distortion”). The distortion of a CU is estimated as a sample value difference, e.g., sum of absolute differences (SAD) or sum of squared differences (SSD). The estimated value obtained from each candidate PU may be used by a mode selector 386 to determine an intra-prediction mode using the differential 324. The prediction mode 387 indicates the decision to use a particular prediction mode, e.g., intra-frame prediction or inter-frame prediction, for the current CU. For intra-predicted CUs belonging to a shared coding tree, independent intra-prediction modes are specified for the luma PB and the chroma PB. For intra-predicted CUs belonging to the luma branch or the chroma branch of a dual coding tree, one intra-prediction mode is applied to the luma PB or the chroma PB, respectively. Estimating the coding cost associated with each candidate prediction mode and the corresponding residual coding can be performed at a significantly lower cost than entropy coding of the residual. Therefore, even in a real-time video encoder, a large number of candidate modes can be evaluated to determine the best one in the rate-distortion sense.

[0083] A Lagrangian or similar optimization process can be employed both to select the optimal partitioning of the CTUs into CBs (by the block partitioner 310) and to select the best prediction mode from multiple possible prediction modes. Application of the Lagrangian optimization process of the candidate modes in the mode selector module 386 selects the intra prediction mode 387, secondary transform index 388, and primary transform type 389, and transform skip flag 390 (one for each TB) that yields the smallest cost measure.

[0084] In the second stage of operation of the video encoder 114 (called the "encoding" stage), the video encoder 114 performs an iteration over the determined coding tree(s) for each CTU. For CTUs using separate trees, for each 64x64 luma region of the CTU, the luma coding tree is coded first, followed by the chroma coding tree. Only the luma CB is coded within the luma coding tree, and only the chroma CB is coded within the chroma coding tree. For CTUs using a shared tree, the CU, i.e., the luma CB and the chroma CB, are described in one tree according to the common block structure of the shared tree.

[0085] The entropy coder 338 supports both variable-length coding of syntax elements and arithmetic coding of syntax elements. Portions of the bitstream, such as "parameter sets," e.g., the sequence parameter set (SPS), picture parameter set (PPS), and picture header (PH), use a combination of fixed-length and variable-length codewords. Slices (also called contiguous portions) have a slice header that uses variable-length coding and slice data that uses arithmetic coding. The picture header defines parameters specific to the current slice, such as picture-level quantization parameter offsets. The slice data contains syntax elements for each CTU within the slice. When using variable-length coding and arithmetic coding, sequential parsing is required within each portion of the bitstream. These portions may be separated by start codes to form "network abstraction layer units" or "NAL units." Arithmetic coding is supported using a context-adaptive binary arithmetic coding process. Arithmetic-coded syntax elements consist of a sequence of one or more "bins." Bins, like bits, have a value of "0" or "1." However, bins are not coded as individual bits in the bitstream 115. Bins have an associated predicted value (or "likely" or "most likely") and associated probability, called a "context." If the actual bin to be coded matches the predicted value, a "most likely symbol" (MPS) is coded. Coding the most likely symbol is relatively inexpensive in terms of consumed bits in the bitstream 115, including the cost of less than one discrete bit. If the actual bin to be coded does not match the likely value, a "least likely symbol" (LPS) is coded. Coding the least likely symbol is relatively expensive in terms of consumed bits. Bin coding techniques can efficiently code bins with biased probabilities of "0" and "1." For syntax elements with two possible values ​​(i.e., "flags"), one bin is sufficient. For syntax elements with many possible values, a series of bins is required.

[0086] The presence of a later bin in the sequence may be determined based on the values ​​of earlier bins in the sequence. Furthermore, each bin may be associated with two or more contexts. The selection of a particular context may depend on the previous bin of the syntax element, the values ​​of bins of adjacent syntax elements (i.e., from adjacent blocks), etc. Each time a context-coded bin is coded, the context (if any) selected for that bin is updated in a manner that reflects the value of the new bin. In this way, binary arithmetic coding schemes are said to be adaptive.

[0087] Also supported by the video encoder 114 are bins without context ("bypass bins"). Bypass bins are coded assuming an equiprobable distribution between "0" and "1". Thus, each bin codes a cost of one bit in the bitstream 115. The absence of context saves memory and reduces complexity; therefore, bypass bins are used where the distribution of values ​​for a particular bin is not skewed. One example of an entropy encoder that employs context and adaptation is known in the art as the Context Adaptive Binary Arithmetic Coder (CABAC), and many variations of this encoder have been employed in video coding.

[0088] The entropy coder 338 codes a secondary transform index 388 using the primary transform type 389, one transform skip flag (i.e., 390) for each TB of the current CU, and, if applicable to the current CU, a combination of context coding and bypass coding bins, and an intra prediction mode 387. The secondary transform index 388 is signaled when the residual associated with the transform block contains significant residual coefficients only at those coefficient positions that are subject to transformation into primary coefficients by application of the secondary transform.

[0089] The multiplexing module 384 outputs the PB 320 from the intra-frame prediction module 364 according to a determined best intra-prediction mode selected from the tested prediction modes of each candidate CB. The candidate prediction modes need not include all possible prediction modes supported by the video encoder 114. Intra-prediction is classified into three types: "DC intra-prediction" involves populating the PB with a single value representing the average of nearby reconstructed samples; "planar intra-prediction" involves populating the PB with samples according to a plane in which the DC offset and vertical and horizontal gradients are derived from nearby reconstructed neighboring samples; the nearby reconstructed samples typically include a row of reconstructed samples above the current PB and extending somewhat to the right of the PB, and a column of reconstructed samples to the left of the current PB and extending somewhat downward beyond the PB; and "angular intra-prediction" involves populating the PB with reconstructed neighboring samples that are filtered and propagated across the PB in a specific direction (or "angle"). VVC supports 65 angles, and rectangular blocks allow for additional angles not available in square blocks, generating a total of 87 angles. For chroma PB, a fourth intra-prediction method, the Cross-Component Linear Model (CCLM) mode, allows for the generation of PB from the collocation of luma reconstructed samples. There are three different CCLM modes, each using a different model derived from adjacent luma and chroma samples. This model is used to generate a block of chroma PB samples from the collocated luma samples.

[0090] When no previously reconstructed samples are available, such as at the edge of a frame, a default halftone value is used, half the range of samples. For example, for 10-bit video, a value of 512 is used. Because there are no previously available samples for the CB at the top-left position of the frame, the angular and planar intra prediction modes produce the same output as the DC prediction mode: a plane of samples with the halftone value as its magnitude.

[0091] In inter-frame prediction, the prediction block 382 is generated by the motion compensation module 380 using samples from one or two frames preceding the current frame in coding order in the bitstream, and is output as the PB 320 by the multiplexing module 384. Furthermore, in inter-frame prediction, a single coding tree is typically used for both the luma and chroma channels. The order of coded frames in the bitstream may differ from the order of frames captured or displayed. When one frame is used for prediction, the block is called "unidirectionally predicted" and has one associated motion vector. When two frames are used for prediction, the block is called "bidirectionally predicted" and has two associated motion vectors. For P slices, each CU is either intra-predicted or unidirectionally predicted. For B slices, each CU is either intra-predicted, unidirectionally predicted, or bidirectionally predicted. Frames are typically coded using a "picture group" structure, allowing for temporal layering of frames. Multiple frames may be divided into multiple slices, each of which codes a portion of the frame. Temporal layering of frames allows frames to reference previous and subsequent images in the order in which they are displayed. Images are coded in the order necessary to ensure that dependencies for decoding each frame are met.

[0092] The samples are selected according to a motion vector 378 and a reference picture index. The motion vector 378 and the reference picture index apply to all color channels. Therefore, inter prediction is primarily described in terms of operations on PUs rather than PBs; that is, the decomposition of each CTU into one or more inter-predicted blocks is described using a single coding tree. Inter prediction may vary in the number and precision of motion parameters. Motion parameters typically consist of a reference frame index indicating which reference frame to use from a list of reference frames and a spatial transformation for each of the reference frames, but may also include more frames, specialized frames, or complex affine parameters such as scaling and rotation. Furthermore, a predetermined motion refinement process may be applied to generate a dense motion estimate based on the referenced sample blocks.

[0093] After determining and selecting PU 320 and subtracting it from the original sample block in subtractor 322, a residual with the lowest coding cost, denoted 324, is obtained and subjected to lossy compression. The lossy compression process consists of transform, quantization, and entropy coding steps. Forward primary transform module 326 applies a forward transform to difference 324, converting it from the time domain to the frequency domain and generating primary transform coefficients, denoted by arrow 328, according to the primary transform type 389. The maximum primary transform size in one dimension is either a 32-point DCT-2 transform or a 64-point DCT-2 transform. If the CB to be coded is larger than the maximum supported primary transform size, denoted as a block size, i.e., 64x64 or 32x32, the primary transform 326 is applied in a tiled manner to transform all samples of difference 324. If each application of the transform operates on a TB larger than 32x32, for example a 64x64 difference 324, then all resulting primary transform coefficients 328 outside the top-left 32x32 region of the TB are set to zero, i.e., discarded. For TBs up to 32x32 in size, the primary transform type 389 can indicate the application of a combination of DST-7 and DCT-8 transforms in the horizontal and vertical directions. The remaining primary transform coefficients 328 are passed to the forward secondary transform module 330.

[0094] The secondary transform module 330 generates secondary transform coefficients 332 according to the secondary transform index 388. The secondary transform coefficients 332 are quantized by module 334 according to a quantization parameter associated with the CB to generate residual coefficients 336. When the transform skip flag 390 indicates that transform skip is enabled for the TB, the difference 324 is passed to the quantizer 334 via multiplexer 333.

[0095] The forward primary transform of module 326 is typically separable, transforming a set of rows of each TB followed by a set of columns. According to the primary transform type 389, forward primary transform module 326 uses either a Type II Discrete Cosine Transform (DCT-2) in the horizontal and vertical directions, or, for the luma TBS, a combination of a Type VII Discrete Sine Transform (DST-7) and a Type VIII Discrete Cosine Transform (DCT-8) in either the horizontal or vertical directions. The use of a combination of DST-7 and DCT-8 is referred to as a "Multiple Transform Selection Set" (MTS) in the VVC standard. When using DCT-2, the maximum TB size is 32x32 or 64x64, configurable by the video encoder 114 and signaled in the bitstream 115. Regardless of the configured maximum DCT-2 transform size, only the coefficients of the top-left 32x32 region of the TB are coded into the bitstream 115. Significant coefficients outside the top-left 32x32 region of the TB are discarded (or "zeroed out") and not coded into the bitstream 115. MTS is only available for CUs up to 32x32 in size, and only coefficients in the top-left 16x16 region of the associated luma TB are coded. Individual TBs of a CU are transformed or bypassed according to the corresponding transform skip flag 390.

[0096] The forward secondary transform of module 330 is generally a non-separable transform, applied only to the residual of intra-predicted CUs, and may nevertheless be bypassed. The forward secondary transform operates on either 16 samples (arranged as the top-left 4x4 sub-block of primary transform coefficients 328) or 48 samples (arranged as three 4x4 sub-blocks of the top-left 8x8 coefficients of primary transform coefficients 328) to generate a set of secondary transform coefficients. The set of secondary transform coefficients may be smaller in number than the set of primary transform coefficients from which they are derived. Due to the application of the secondary transform only to sets of coefficients that are adjacent to each other and include the DC coefficient, the secondary transform is referred to as a "low-frequency non-separable secondary transform" (LFNST).

[0097] The residual coefficients 336 are provided to an entropy encoder 338 for encoding in the bitstream 115. Typically, the residual coefficients of each TB having at least one significant residual coefficient of a TU are scanned to generate an ordered list of values ​​according to a scan pattern. The scan pattern generally scans the TB as a sequence of 4x4 "sub-blocks," providing a regular scan operation at the granularity of 4x4 sets of residual coefficients, with the placement of the sub-blocks depending on the size of the TB. The scan within each sub-block and the progression from one sub-block to the next typically follows a backward diagonal scan pattern.

[0098] As mentioned above, the video encoder 114 requires access to a frame representation that corresponds to the decoded frame representation seen by the video decoder 134. Accordingly, the residual coefficients 336 are passed to a dequantizer 340 to generate dequantized residual coefficients 342. The dequantized residual coefficients 342 are passed to an inverse secondary transform module 344, which operates according to secondary transform indexes 388, to generate intermediate inverse transform coefficients, represented by arrow 346. The intermediate inverse transform coefficients 346 are passed to an inverse primary transform module 348 to generate residual samples, represented by TU arrow 399. The quantized residual coefficients 342 are output by a multiplexer 349 as residual samples 350 if a transform skip 390 indicates that a transform bypass is to be performed. Otherwise, the multiplexer 349 outputs the residual samples 399 as residual samples 350.

[0099] The type of inverse transform performed by the inverse secondary transform module 344 corresponds to the type of forward transform performed by the forward secondary transform module 330. The type of inverse transform performed by the inverse primary transform module 348 corresponds to the type of primary transform performed by the primary transform module 326. A summation module 352 sums the residual samples 350 and the PU 320 to generate reconstructed samples for the CU (indicated by arrow 354).

[0100] The reconstructed samples 354 are passed to a reference sample cache 356 and an in-loop filter module 368. The reference sample cache 356 is typically implemented using static RAM on the ASIC (thus avoiding costly off-chip memory accesses) and provides the minimum sample storage necessary to satisfy dependencies for generating intra-frame PBs for subsequent CUs in the frame. These dependencies include a "line buffer" of samples along the bottom of a row of CTUs for use by the next row of CTUs, as well as a column buffer set by the height of the CTU. The reference sample cache 356 supplies reference samples (represented by arrow 358) to a reference sample filter 360. The sample filter 360 applies a smoothing operation to generate filtered reference samples (indicated by arrow 362). The filtered reference samples 362 are used by an intra-frame prediction module 364 to generate an intra-predicted block of samples, represented by arrow 366. For each candidate intra-prediction mode, the intra-frame prediction module 364 generates a block of samples 366. The block of samples 366 is generated by the module 364 using techniques such as DC, planar or angular intra prediction according to an intra prediction mode 387 .

[0101] The in-loop filter module 368 applies several filtering stages to the reconstructed samples 354. The filtering stages include a "deblocking filter" (DBF), which applies smoothing aligned with CU boundaries to reduce artifacts resulting from discontinuities. Another filtering stage present in the in-loop filter module 368 is an "adaptive loop filter" (ALF), which applies a Wiener-based adaptive filter to further reduce distortion. Another filtering stage present in the in-loop filter module 368 is a "sample adaptive offset" (SAO) filter. The SAO filter operates by first classifying the reconstructed samples into one or more categories and then applying an offset at the sample level depending on the assigned category.

[0102] The filtered samples, represented by arrow 370, are output from the in-loop filter module 368. The filtered samples 370 are stored in a frame buffer 372. The frame buffer 372 typically has the capacity to store multiple pictures (e.g., up to 16) and is therefore stored in the memory 206. The frame buffer 372 is typically not stored using on-chip memory due to the large memory consumption required. As such, accessing the frame buffer 372 is costly in terms of memory bandwidth. The frame buffer 372 provides reference frames (represented by arrow 374) to a motion estimation module 376 and a motion compensation module 380.

[0103] The motion estimation module 376 references one block of a reference frame in the frame buffer 372 to estimate multiple “motion vectors” (denoted as 378), each of which is a Cartesian spatial offset from the position of the current CB. A filtered block of reference samples (denoted as 382) is generated for each motion vector. The filtered reference samples 382 form additional candidate modes available for potential selection by the mode selector 386. Furthermore, for a given CU, the PU 320 may be formed using one reference block (“unidirectional prediction”) or two reference blocks (“bidirectional prediction”). For the selected motion vector, the motion compensation module 380 generates the PB 320 according to a filtering process that supports sub-pixel accuracy of the motion vector. In this way, the motion estimation module 376 (operating with many candidate motion vectors) can perform a simplified filtering process, achieving reduced computational complexity, compared to the filtering process of the motion compensation module 380 (operating with only the selected candidate). When video encoder 114 selects inter prediction for a CU, motion vector 378 is coded into bitstream 115.

[0104] 3 is described with reference to Versatile Video Coding (VVC), other video coding standards or implementations may employ the processing stages of modules 310-386. Also, frame data 113 (and bitstream 115) may be read from (or written to) memory 206, hard disk drive 210, CD-ROM, Blu-ray Disc™, or other computer-readable storage medium. Furthermore, frame data 113 (and bitstream 115) may be received from (or transmitted to) an external source, such as a server or radio frequency receiver connected to communications network 220.

[0105] Video decoder 134 is shown in FIG. 4. Video decoder 134 of FIG. 4 is an example of a Versatile Video Coding (VVC) video decoding pipeline, although other video codecs may be used to perform the processing stages described herein. As shown in FIG. 4, video decoder 134 receives a bitstream 133 as input. Bitstream 133 may be read from memory 206, hard disk drive 210, CD-ROM, Blu-ray Disc™, or other non-transitory computer-readable storage medium. Alternatively, bitstream 133 may be received from an external source, such as a server or radio frequency receiver connected to communications network 220. Bitstream 133 includes encoded syntax elements representing captured frame data to be decoded.

[0106] The bitstream 133 is input to the entropy decoder module 420. The entropy decoder module 420 extracts syntax elements from the bitstream 133 by decoding a sequence of "bins" and passes the syntax element values ​​to other modules in the video decoder 134. The entropy decoder module 420 uses variable-length and fixed-length decoding to decode the slice data syntax elements as a sequence of one or more bins using the SPS, PPS, or slice header arithmetic decoding engine. Each bin can use one or more "contexts," which describe the probability levels used to code the bin's "1" and "0" values. If multiple contexts are available for a given bin, a "context modeling" or "context selection" step is performed to select one of the available contexts to decode the bin.

[0107] The entropy decoder module 420 applies an arithmetic coding algorithm, such as "context-adaptive binary arithmetic coding" (CABAC), to decode syntax elements from the bitstream 133. The decoded syntax elements are used to reconstruct parameters in the video decoder 134. The parameters include residual coefficients (represented by arrow 424), quantization parameters (not shown), secondary transform indices 474, and mode selection information (represented by arrow 458), such as intra-prediction modes. The mode selection information also includes information such as motion vectors and the partitioning of each CTU into one or more CUs. The parameters are typically used in combination with sample data from a previously decoded CB to generate the PB.

[0108] The residual coefficients 424 are passed to an inverse quantization module 428, which performs inverse quantization (or "scaling") on the residual coefficients 424 (i.e., the primary transform coefficient domain) to create reconstructed transform coefficients, represented by arrow 432, according to a quantization parameter. The reconstructed transform coefficients 432 are passed to an inverse secondary transform module 436, which performs either a secondary transform or no operation (bypass) according to a secondary transform type 474 decoded from the bitstream 113 by the entropy decoder 420 according to the methods described with reference to Figures 15 and 16. The inverse secondary transform module 436 generates reconstructed transform coefficients 440 (i.e., the primary transform domain coefficients).

[0109] The reconstructed transform coefficients 440 are passed to an inverse primary transform module 444. The module 444 inversely transforms the coefficients 440 from the frequency domain to the spatial domain according to the primary transform type 476 (or "mts_idx") decoded from the bitstream 133 by the entropy decoder 420. The result of the operation of the module 444 is a block of residual samples, represented by an arrow 499. If the transform skip flag 478 for a given TB of the CU indicates a bypass of the transform, the multiplexer 449 outputs the reconstructed transform coefficients 432 to the summation module 450 as residual samples 488. Otherwise, the multiplexer 449 outputs residual samples 499 as residual samples 488. The residual samples 448 are equal in size to the corresponding CB. The residual samples 448 are provided to the summation module 450. In summation module 450, residual samples 448 are added to the decoded PB (represented as 452) to produce a block of reconstructed samples represented by arrow 456. The reconstructed samples 456 are provided to a reconstructed sample cache 460 and to an in-loop filtering module 488. The in-loop filtering module 488 produces a reconstructed block of frame samples represented by 492. The frame samples 492 are written to a frame buffer 496 from which the frame data 135 is later output.

[0110] The reconstructed sample cache 460 operates similarly to the reconstructed sample cache 356 of the video encoder 114. The reconstructed sample cache 460 provides storage of the reconstructed samples needed to intra-predict a subsequent CB without relying on access to the memory 206 (e.g., by using the data 232, a typical on-chip memory, instead). Reference samples, represented by arrow 464, are obtained from the reconstructed sample cache 460 and provided to a reference sample filter 468 to generate filtered reference samples, represented by arrow 472. The filtered reference samples 472 are provided to an intra-frame prediction module 476. The module 476 generates blocks of intra-predicted samples, represented by arrow 480, according to the intra-prediction mode parameters 458 signaled in the bitstream 133, which are decoded by the entropy decoder 420.

[0111] If the prediction mode of a CB is indicated to use intra prediction in the bitstream 133, the intra-predicted samples 480 form the decoded PB 452 via the multiplexing module 484. Intra prediction generates a predictive block of samples (PB), i.e., a block in one color component derived using "neighboring samples" in the same color component. Neighboring samples are samples adjacent to the current block and have already been reconstructed by preceding them in block decoding order. When luma and chroma blocks are arranged, the luma and chroma blocks can use different intra prediction modes. However, two chroma CBs share the same intra prediction mode.

[0112] If the prediction mode of the CB is indicated as inter prediction in the bitstream 133, the motion compensation module 434 uses the motion vector (decoded from the bitstream 133 by the entropy decoder 420) and the reference frame index to select and filter a block of samples 498 from a frame buffer 496 to generate a block of inter predicted samples represented as 438. The block of samples 498 is obtained from a previously decoded frame stored in the frame buffer 496. For bidirectional prediction, two blocks of samples are generated and blended together to generate samples for the decoded PB 452. The frame buffer 496 receives filtered block data 492 from the in-loop filtering module 488. Similar to the in-loop filtering module 368 of the video encoder 114, the in-loop filtering module 488 applies any of the DBF, ALF, and SAO filtering operations. Generally, motion vectors are applied to both the luma and chroma channels, but the filtering processes for sub-sample interpolation in the luma and chroma channels are different.

[0113] 5 is a schematic block diagram illustrating a collection 500 of available partitions or partitions that divide a region into one or more subregions at each node of a coding tree structure for versatile video coding. The partitions shown in collection 500 are available to block partitioner 310 of encoder 114 to divide each CTU into one or more CUs or CBs according to the coding tree as determined by Lagrangian optimization, as described with reference to FIG.

[0114] Although collection 500 only illustrates the division of square regions into other, possibly non-square, sub-regions, it should be understood that collection 500 illustrates the possibility of dividing a parent node of a coding tree into child nodes of the coding tree, and does not require that the parent node correspond to a square region. If the contained region is non-square, the size of the blocks resulting from the division is scaled according to the aspect ratio of the contained blocks. When a region is not further divided, i.e., at a leaf node of the coding tree, a CU occupies the region.

[0115] The process of subdividing a region into subregions ends when the resulting subregions reach a minimum CU size (typically 4x4 luma samples). CUs are constrained to prohibit block regions smaller than a certain minimum size (e.g., 16 samples), as well as a minimum width or height of 4. Alternatively, minimums for both width and height, or both width and height, may be set. The subdivision process may end before the deepest level of decomposition, resulting in a CU larger than the minimum CU size. It is also possible for no subdivision to occur, resulting in a single CU occupying the entire CTU. A single CU occupying the entire CTU represents the largest available coding unit size. The use of subsampled chroma formats, such as 4:2:0, may allow the video encoder 114 and video decoder 134 arrangement to complete the division of regions for the chroma channels earlier than for the luma channel, including in the case of a shared coding tree that defines the block structure for the luma and chroma channels. When separate coding trees are used for luma and chroma, constraints on the available splitting operations ensure that the minimum chroma CU area is 16 samples, even if such a CU is co-located with a larger luma area, e.g., 64 luma samples.

[0116] At the leaf nodes of the coding tree, there are CUs. For example, leaf node 510 contains one CU. At the non-leaf nodes of the coding tree, there are divisions into two or more further nodes, each of which may contain a leaf node forming one CU or a non-leaf node containing a further division into smaller regions. At each leaf node of the coding tree, there is one CB for each color channel of the coding tree. A division that ends at the same depth for both luma and chroma in the shared tree results in one CU having three conjugate CBs.

[0117] The quadtree partition 512 divides the containing region into four equally sized regions, as shown in Figure 5. Compared to HEVC, Versatile Video Coding (VVC) achieves further flexibility through additional partitions, including a horizontal bisection 514 and a vertical bisection 516. Each of the partitions 514 and 516 divides the contained region into two equally sized regions. The partitions occur along either a horizontal boundary (514) or a vertical boundary (516) within the containing block.

[0118] Versatile video coding provides further flexibility by adding a 3-way horizontal partition 518 and a 3-way vertical partition 520. The 3-way partitions 518 and 520 divide a block into three regions bounded either horizontally (518) or vertically (520) along ¼ and ¾ of the width or height of the containing region. The combination of a quadtree, a binary tree, and a ternary tree is called a "QTBTTT." The root of the tree contains zero or more quadtree partitions (the "QT" section of the tree). Once the QT section ends, zero or more bipartitions or tripartites occur (the "multitree" or "MT" section of the tree), eventually terminating in a CB or CU at the tree's leaf node. If the tree describes all color channels, the tree's leaf node is a CU. If the tree describes the luma or chroma channels, the tree's leaf node is a CB.

[0119] Compared to HEVC, which only supports quadtrees and therefore only square blocks, QTBTTT offers many more possible CU sizes, especially when considering the possibility of recursively applying binary and / or ternary tree partitioning. When only quadtree partitioning is available, each increase in coding tree depth corresponds to a reduction in CU size by one-quarter of the parent area. In VVC, because binary and ternary tree partitioning is possible, coding tree depth no longer directly corresponds to CU area. Restricting partitioning options to eliminate partitions where the block width or height is less than four samples or is not a multiple of four samples reduces the likelihood of non-square block sizes. Restricting partitioning options to eliminate partitions where the block width or height is less than four samples or is not a multiple of four samples reduces the likelihood of unusual (non-square) block sizes.

[0120] 6 is a schematic flow diagram illustrating a data flow 600 of a QTBTTT (or "coding tree") structure used in versatile video coding. The QTBTTT structure is used for each CTU to define the partitioning of the CTU into one or more ECUs. The QTBTTT structure for each CTU is determined by a block partitioner 310 in the video encoder 114 and is encoded into or decoded from the bitstream 115 by an entropy decoder 420 in the video decoder 134. The data flow 600 further features the allowable combinations available to the block partitioner 310 for partitioning the CTU into one or more CUs according to the partitioning shown in FIG. 5.

[0121] Starting from the top level of the hierarchy, i.e., CTU, zero or more quad-tree splits are first performed. Specifically, a quad-tree (QT) split decision 610 is made by the block partitioner 310. A decision at 610 returning a "1" symbol indicates a decision to split the current node into four subnodes according to the quad-tree split 512. As a result, four new nodes are generated, as at 620, and for each new node, a recursion is made to the QT split decision 610. Each new node is considered in raster (or Z-scan) order. Alternatively, if the QT split decision 610 indicates no further splits (returning a "0" symbol), the quad-tree split stops and a multi-tree (MT) split is subsequently considered.

[0122] First, an MT split decision 612 is made by the block partitioner 310. At 612, a decision is indicated as to whether to perform MT splitting. Returning a "0" symbol at decision 612 indicates that no further splitting of the node into subnodes is to be performed. If no further splitting of the node is to be performed, the node is a leaf node of the coding tree and corresponds to a CU. The leaf node is output at 622. Alternatively, if MT split 612 indicates a decision to perform MT splitting (returning a "1" symbol), the block partitioner 310 proceeds to direction decision 614.

[0123] The direction decision 614 indicates the direction of the MT partition as either horizontal ("H" or "0") or vertical ("V" or "1"). The block partitioner 310 proceeds to decision 616 if decision 614 returns a "0" indicating a horizontal direction. The block partitioner 310 proceeds to decision 618 if decision 614 returns a "1" indicating a vertical direction.

[0124] In each of decisions 616 and 618, the number of divisions for the MT division is indicated as either two (bisection or "BT" node) or three (trisection or "TT") for the BT / TT division. That is, if the direction pointed from 614 is horizontal, then BT / TT division decision 616 is made by block partitioner 310, and if the direction pointed from 614 is vertical, then BT / TT division decision 618 is made by block partitioner 310.

[0125] The BT / TT split decision 616 indicates whether the horizontal split is a 2-way split 514, indicated by returning a "0," or a 3-way split 518, indicated by returning a "1." If the BT / TT split decision 616 indicates a 2-way split, then in a generate HBT CTU node step 625, two nodes are generated by the block partitioner 310 according to the horizontal 2-way split 514. When the BT / TT split 616 indicates a 3-way split, then in a generate HTT_CTU node step 626, three nodes are generated by the block partitioner 310 according to the horizontal 3-way split 518.

[0126] The BT / TT split decision 618 indicates whether the vertical split is a 2-way split 516, indicated by returning a "0," or a 3-way split 520, indicated by returning a "1." If the BT / TT split 618 indicates a 2-way split, then in a VBT_CTU node generation step 627, the block partitioner 310 generates two nodes according to the vertical 2-way split 516. If the BT / TT split 618 indicates a 3-way split, then in a VTT_CTU node generation step 628, the block partitioner 310 generates three nodes according to the vertical 3-way split 520. For each node resulting from steps 625-628, a recursion of the data flow 600 back to the MT split decision 612 is applied in a left-to-right or top-to-bottom order, depending on the direction 614. As a result, binary and ternary tree splits can be applied to generate CUs with various sizes.

[0127] 7A and 7B provide an example 700 of partitioning a CTU 710 into multiple CUs or CBs. An example CU 712 is shown in Figure 7A. Figure 7A also shows the spatial arrangement of the CUs in the CTU 710. The example partition 700 is also shown as a coding tree 720 in Figure 7B.

[0128] At each non-leaf node of the CTU 710 in FIG. 7A, e.g., nodes 714, 716, and 718, the contained nodes (which may be further split or may be CUs) are scanned or traversed in "Z-order" to create a list of nodes represented as columns in the coding tree 720. For quadtree splits, the Z-order scan is performed from the top left to the right, followed by the bottom left to the right. For horizontal and vertical splits, the Z-order scan (traversal) is simplified to a top-to-bottom scan and a left-to-right scan, respectively. The coding tree 720 in FIG. 7B lists all nodes and CUs ordered according to the Z-order scan of the coding tree. Each split generates a list of two, three, or four new nodes at the next level of the tree until a leaf node (CU) is reached.

[0129] After the block partitioner 310 decomposes the image into CTUs and then into CUs to generate respective residual blocks (324) using the CUs as described with reference to FIG. 3, the residual blocks are subjected to forward transform and quantization by the video encoder 114. The resulting TBs 336 are then scanned to form a sequential list of residual coefficients as part of the operation of the entropy coding module 338. An equivalent process is performed in the video decoder 134 to obtain the TBs from the bitstream 133.

[0130] 8A, 8B, 8C, and 8D show examples of forward and inverse non-separable secondary transforms performed according to transform block (TB) sizes of different sizes. FIG. 8A illustrates a set of relationships 800 between primary transform coefficients 802 and secondary transform coefficients 804 for a 4×4 TB size. The primary transform coefficients 802 are composed of 4×4 coefficients, and the secondary transform coefficients 804 are composed of eight coefficients. The eight secondary transform coefficients are arranged in a pattern 806. The pattern 806 corresponds to eight adjacent positions in a backward diagonal scan of the TB, including the DC (top-left) position. The remaining eight positions in the backward diagonal scan shown in FIG. 8A are not input by performing a forward secondary transform and therefore remain zero-valued. Therefore, a forward non-separable secondary transform 810 for a 4×4 TB receives 16 primary transform coefficients and produces eight secondary transform coefficients as output. Therefore, the forward secondary transform 810 for a 4×4 TB can be represented by an 8×16 matrix of weights. Similarly, the inverse quadratic transform 812 can be represented by a 16x8 matrix of weights.

[0131] 8B shows a set of relationships 818 between primary and secondary transform coefficients for 4×N and N×4 TB sizes (N is greater than 4), in both cases where the top-left 4×4 sub-block of primary coefficients 820 is associated with the top-left 4×4 sub-block of secondary transform coefficients 824. In the video encoder 114, a forward non-separable secondary transform 830 takes 16 primary transform coefficients and produces 16 secondary transform coefficients as output. The remaining primary transform coefficients 822 are not populated by the forward secondary transform and therefore remain at zero values. After the forward non-separable secondary transform 830 is performed, coefficient positions 826 associated with coefficients 822 are not populated and therefore remain at zero values.

[0132] A forward quadratic transform 830 for a 4xN or Nx4 TB can be represented by a 16x16 matrix of weights. The matrix representing the forward quadratic transform 830 is defined as A. Similarly, a corresponding inverse quadratic transform 832 can be represented by a 16x16 matrix of weights. The matrix representing the inverse quadratic transform 832 is defined as B.

[0133] The storage requirements of the non-separable transform kernel are further reduced by reusing portions of A for the forward secondary transform 810 and the inverse secondary transform 812 to 4x4TB. The first 8 rows of A are used for the forward secondary transform 810, and the transpose of the first 8 rows of A is used for the inverse secondary transform 812.

[0134] FIG. 8C illustrates a relationship 855 between primary transform coefficients 840 and secondary transform coefficients 842 for a TB of size 8x8. The primary transform coefficients 840 are composed of 8x8 coefficients, and the secondary transform coefficients 842 are composed of eight transform coefficients. The eight secondary transform coefficients 842 are arranged in a pattern corresponding to eight consecutive positions in a backward diagonal scan of the TB, where the eight consecutive positions include the DC (top-left) coefficient of the TB. The remaining secondary transform coefficients of the TB are all zero and therefore do not need to be scanned. A forward unsplit secondary transform 850 for an 8x8 TB takes as input 48 primary transform coefficients corresponding to three 4x4 sub-blocks and generates eight secondary transform coefficients. The forward secondary transform 850 for an 8x8 TB can be represented by an 8x48 matrix of weights. The corresponding inverse secondary transform 852 for an 8x8 TB can be represented by a 48x8 matrix of weights.

[0135] 8D illustrates a relationship 875 between primary transform coefficients 860 and secondary transform coefficients 862 for a TB of size 8x8 or larger. The top-left 8x8 block of primary coefficients 860 (arranged as four 4x4 sub-blocks) is associated with the top-left 4x4 sub-block of secondary transform coefficients 862. In the video encoder 114, a forward non-separable secondary transform 870 operates on the 48 primary transform coefficients to generate 16 secondary transform coefficients. The remaining primary transform coefficients 864 are zeroed. Secondary transform coefficient positions 866 outside the top-left 4x4 sub-block of secondary transform coefficients 862 are not populated and remain zero.

[0136] The forward quadratic transform 870 for TBs of size greater than 8x8 can be represented by a 16x48 matrix of weights. The matrix representing the forward quadratic transform 870 is defined as F. Similarly, the corresponding inverse quadratic transform 832 can be represented by a 48x16 matrix of weights. The matrix representing the inverse quadratic transform 872 is defined as G. As discussed above with reference to matrices A, B, and F, it preferably has the property of orthogonality, where G=F. T Only F needs to be stored in the video encoder 114 and the video decoder 134. An orthogonal matrix can be expressed as a matrix whose rows have orthogonality.

[0137] The storage requirements of the non-separable transform kernels are further reduced by reusing parts of F. F is for the 8x8TB forward secondary transform 850 and inverse secondary transform 852. The first 8 rows of F are used for the forward secondary transform 810, while the transpose of the first 8 rows of F is used for the inverse secondary transform 812.

[0138] A non-separable quadratic transform can sparsify two-dimensional features of the residual signal, such as angular features, thereby achieving coding improvements over using only a separable linear transform. Because angular features in the residual signal may depend on the type of intra prediction mode 387 selected, it is advantageous to adaptively select a non-separable quadratic transform matrix depending on the intra prediction mode. As described above, intra prediction modes consist of "intra DC" mode, "intra plane" mode, "intra angle" mode, and "matrix intra prediction" mode. The intra prediction mode parameter 458 takes a value of 0 when intra DC prediction is used. The intra prediction mode parameter 458 takes a value of 1 when intra plane prediction is used. The intra prediction mode parameter 458 takes a value between 2 and 66 when intra angular prediction on a square TB is used.

[0139] Figure 9 illustrates a set 900 of transform blocks available in the Versatile Video Coding (VVC) standard. Figure 9 also illustrates the application of a secondary transform to a subset of residual coefficients from the transform blocks of set 900. Figure 9 shows multiple TBs ranging in width and height from 4 to 32. However, TBs with a width and / or height of 64 are possible but are not shown for ease of reference.

[0140] A 16-point quadratic transform 952 (shown in dark shading) is applied to a set of 4x4 coefficients. The 16-point quadratic transform 952 is applied to TBs with a width or height of 4, such as 4x4 TB 910, 8x4 TB 912, 16x4 TB 914, 32x4 TB 916, 4x8 TB 920, 4x16 TB 930, and 4x32 TB 940. The 16-point quadratic transform 952 is also applied to TBs of size 4x64 and 64x4 (not shown in FIG. 9). For TBs with a width or height of 4 and 16 or more primary coefficients, the 16-point quadratic transform is applied only to the top-left 4x4 sub-block of the TB; other sub-blocks require zero-valued coefficients for the secondary transform to be applied. Applying a 16-point quadratic transform generally results in 8 or 16 secondary transform coefficients, as described with reference to FIGS. 8 through 8D. The secondary transform coefficients are packed into the TB for encoding in the top left sub-block of the TB.

[0141] For transform sizes greater than 4 in width and height, a 48-point quadratic transform 950 (shown in light shading) is available for application to three 4x4 sub-blocks of residual coefficients in the top-left 8x8 region of the transform block, as shown in Figure 9. The 48-point quadratic transform 950 is applied to an 8x8 transform block 922, a 16x8 transform block 924, a 32x8 transform block 926, an 8x16 transform block 932, a 16x16 transform block 934, a 32x16 transform block 936, an 8x32 transform block 942, a 16x32 transform block 944, and a 32x32 transform block 946, in each case in the regions shown in light shading and dashed lines. The 48-point quadratic transform 950 can also be applied to TBs (not shown) of sizes 8x64, 16x64, 32x64, 64x64, 64x32, 64x16, and 64x8. Applying a 48-point secondary transform kernel generally results in fewer than 48 secondary transform coefficients. For example, as described with reference to Figures 8B-8D, 8 or 16 secondary transform coefficients may be generated. Primary transform coefficients not subject to the secondary transform ("primary-only coefficients"), e.g., coefficient 966 of TB934, are required to have a zero value in order for the secondary transform to be applied. After applying the 48-point secondary transform 950 in the forward direction, the region that may contain significant coefficients is reduced from 48 coefficients to 16 coefficients, further reducing the number of coefficient positions that may contain significant coefficients. In the inverse secondary transform, the decoded significant coefficients are transformed to generate coefficients that may be significant in the region that is subject to the primary inverse transform. When the secondary transform reduces one or more sub-blocks to a set of 16 secondary transform coefficients, only the top-left 4x4 sub-block may contain significant coefficients. The location of the last significant coefficient in any coefficient position where a secondary transform coefficient may be stored indicates the application of a secondary transform or only a primary transform.

[0142] When the last significant coefficient position indicates a secondary transform coefficient position within the TB, a signaled secondary transform index (i.e., 388 or 474) is needed to distinguish whether to apply a secondary transform kernel or bypass the secondary transform. Although the application of the secondary transform to TBs of various sizes in FIG. 9 has been described from the perspective of the video encoder 114, the corresponding inverse processing is performed in the video decoder 134. The video decoder 134 first decodes the position of the last significant coefficient. If the decoded last significant coefficient position indicates the applicability of a secondary transform, the secondary transform index 474 is decoded to determine whether to apply or bypass the inverse secondary transform.

[0143] FIG. 10 shows a syntax structure 0100 for a bitstream 1001 having multiple slices. Each slice contains multiple coding units. The bitstream 1001 may be generated by a video encoder 114, e.g., as bitstream 115, or parsed by a video decoder 134, e.g., as bitstream 133. The bitstream 1001 is divided into portions, e.g., network abstraction layer (NAL) units, and the division is achieved by preceding each NAL unit with a NAL unit header, e.g., 1008. A sequence parameter set (SPS) 1010 defines sequence-level parameters, such as the profile (set of tools), chroma format, sample bit depth, and frame resolution, used to encode and decode the bitstream. Parameters are also included in the set 1010 that constrain the application of different types of divisions in the coding tree of each CTU.

[0144] The Picture Parameter Set (PPS) 1012 defines a set of parameters that apply to zero or more frames. The Picture Header (PH) 1015 defines the parameters that apply to the current frame. The parameters in the PH 1015 may include a list of CU chroma QP offsets, one of which can be applied at the CU level to derive the quantization parameters for use by a chroma block from the quantization parameters of co-occurring luma CBs.

[0145] A picture header 1015 and a sequence of slices that form a picture are known as AUs (Access Units), such as AU0_0 114. AU0_1014 contains three slices, such as slices 0 to 2, with slice 1 being denoted 1016. Like the other slices, slice 1 (1016) contains a slice header 1018 and slice data 1020.

[0146] FIG. 11 shows a syntax structure 1100 for slice data (e.g., 1020 and corresponding slice data 1104) of bitstream 1001 (e.g., 115 and 133) according to a shared coding tree of luma and chroma coding units of a coding tree unit, such as CTU 1110. CTU 1110 includes one or more CUs. An example is labeled CU 1114. CU 1114 includes a signaled prediction mode 1116 followed by a transform tree 1118. When the size of CU 1114 does not exceed the maximum transform size (either 32x32 or 64x64 for the luma channel), transform tree 1118 includes one transform unit, denoted as TU 1124. When a 4:2:0 chroma format is used, the corresponding maximum chroma transform size is half the luma maximum transform size in each direction. That is, if the maximum luma transform size is 32x32 or 64x64, the maximum chroma transform size is 16x16 or 32x32, respectively. For 4:4:4 chroma formats, the maximum chroma transform size is the same as the maximum luma transform size. For 4:2:2 chroma formats, the maximum chroma transform size is half the horizontal size and equal to the maximum luma transform size vertically. That is, if the maximum luma transform size is 32x32 or 64x64, the maximum chroma transform size is 16x32 or 32x64, respectively.

[0147] If the prediction mode 1116 indicates the use of intra prediction for the CU 1114, a luma intra prediction mode and a chroma intra prediction mode are specified. Additionally, for the luma CB of the CU 1114, the primary transform type is signaled as either (i) DCT-2 horizontally and vertically, (ii) transform skip horizontally and vertically, or (iii) a combination of DST-7 and DCT-8 horizontally and vertically according to the MTS index 1122. If the signaled luma transform type is DCT-2 horizontally and vertically (option (i)), an additional luma secondary transform index 1120, also known as a “low-frequency non-separable transform” (LFNST) index, is signaled in the bitstream under the conditions as described with reference to Figures 8A-8D and 13-16.

[0148] By using a shared coding tree, the TU 1124 includes a TB for each color channel, denoted as luma TB_Y_1128, first chroma TB_Cb_1132, and second chroma TB_Cr_1136. The presence of each TB depends on its corresponding "coded block flag" (CBF), i.e., one of the coded block flags 1123. When a TB is present, the corresponding CBF is equal to one, and at least one residual coefficient in the TB is nonzero. When a TB is absent, the corresponding CBF is equal to zero, and all residual coefficients in the TB are zero. The luma TB 1128, first chroma TB 1134, and second chroma TB 1136 may use transform skips, as signaled by the transform skip flags 1126, 1130, and 1134, respectively. A coding mode in which a single chroma TB is transmitted to specify the chroma residual for both the Cb and Cr channels is available, known as a "joint CbCr" coding mode. If the joint CbCr coding mode is enabled, a single chroma TB is coded.

[0149] Regardless of the color channel, each coded TB includes one or more residual coefficients following the final position. For example, luma TB 1128 includes final position 1140 and residual coefficient 1144. Final position 1140 indicates the position of the last significant residual coefficient in the TB when considering the coefficients of the diagonal scan pattern used to serialize the array of coefficients of the TB in a forward direction (i.e., starting from the DC coefficient). Two TBs 1132 and 1136 for the chroma channels each have a corresponding final position syntax element used in a manner similar to that described for luma TB 1128. If the final positions of the TBs for a CU, i.e., 1128, 1132, and 1136, indicate that only coefficients in the secondary transform domain are significant for the respective TBs of the CU, and the remaining coefficients that will undergo only a primary transform are all zero, then a secondary transform index 1120 may be signaled to specify whether to apply a secondary transform. Further conditions regarding the signaling of the secondary transform index 1120 are described with reference to Figures 14 and 16.

[0150] If a secondary transform is applied, the secondary transform index 1120 indicates which kernel is selected. Generally, two kernels are available in a "candidate set" of kernels. Generally, there are four candidate sets, and one candidate set is selected using the intra prediction mode of the block. The luma intra prediction mode is used to select the candidate set for the luma block, and the chroma intra prediction mode is used to select the candidate set for the two chroma blocks. As described with reference to FIGS. 8A-8D, the selected kernel also depends on the TB size, with different kernels used for 4x4, 4xN / Nx4, and other sized TBs. When a 4:2:0 chroma format is used, the chroma TB is generally half the width and height of the corresponding luma TB, resulting in different selected kernels for the chroma blocks when a luma TB with a width or height of 8 is used. For luma blocks of size 4x4, 4x8, and 8x4, the one-to-one correspondence between luma blocks and chroma blocks in the shared coding tree is changed so that there are no chroma blocks of smaller sizes such as 2x2, 2x4, and 4x2.

[0151] The secondary transform index 1120 indicates, for example, the following: index values ​​0 (do not apply), 1 (apply the first kernel from the candidate set), and 2 (apply the second kernel from the candidate set). For chroma, a selected secondary transform kernel from the candidate set, derived considering the chroma TB size and the chroma intra prediction mode, is applied to each chroma channel. Therefore, the residuals of the Cb block 1224 and the Cr block 1226 are required to contain only significant coefficients at positions that undergo secondary transform, as described with reference to FIGS. 8A-8D. When joint CbCr coding is used, the resulting Cb and Cr residuals only contain significant coefficients at positions that correspond to significant coefficients in the jointly coded TB, so the requirement of containing significant coefficients at positions that are subject to secondary transform is only applicable to single-coded chroma TBs.

[0152] FIG. 12 illustrates a syntax structure 1200 for slice data 1204 (e.g., 1020) of a bitstream (e.g., 115, 133) in which the luma coding unit and chroma coding unit of a coding tree unit have separate coding trees. Separate coding trees are available for an "I-slice." The slice data 1204 includes one or more CTUs, such as CTU 1210. CTU 1210 typically has a luma sample size of 128x128 and starts with a shared tree that includes a single quadtree partition common to luma and chroma. In each of the resulting 64x64 nodes, separate coding trees start for luma and chroma. An example of node 1214 is illustrated in FIG. 12. Node 1214 includes luma node 1214a and chroma node 1214b. The luma tree starts with luma node 1214a, and the chroma tree starts with chroma node 1214b. Because the trees continuing from node 1214a and node 1214b are independent between luma and chroma, different splitting options are possible for generating the resulting CUs. Luma CU 1220 belongs to the luma coding tree and includes a luma prediction mode 1221, a luma transform tree 1222, and a secondary transform index 1224. Luma transform tree 1222 includes TU 1230. Because the luma coding tree encodes only luma channel samples, TU 1230 includes luma TB 1234, and luma transform skip flag 1232 indicates whether the luma residual should be transformed. Luma TB 1234 includes a final position 1236 and residual coefficients 1238.

[0153] Chroma CU 1250 belongs to a chroma coding tree and includes a chroma prediction mode 1251, a chroma transform tree 1252, and a secondary transform index 1254. Chroma transform tree 1252 includes TU 1260. Because the chroma tree includes chroma blocks, TU 1260 includes Cb_TB 1264 and Cr_TB 1268. The application of transform bypass to Cb_TB 1264 and Cr_CB 1268 is signaled by Cb transform skip flag 1262 and Cr transform skip flag 1266, respectively. Each TB includes a final position and a residual coefficient; for example, final position 1270 and residual coefficient 1272 are associated with Cb_TB 1264. Signaling of secondary transform index 1254 applied to chroma TBs of the chroma tree is described with reference to Figures 14 and 16.

[0154] FIG. 17 illustrates a 32×32 TB 1700. A conventional scan pattern 1710 is shown applied to the TB 1700. The scan pattern 1710 progresses diagonally backward through the TB 1700, starting at the last significant coefficient position and progressing toward the DC (upper left) coefficient position. This progression divides the TB 1700 into 4×4 sub-blocks. Each sub-block is internally scanned diagonally backward, as shown in some sub-blocks of the TB 1700, such as sub-block 1750. Other sub-blocks are scanned in a similar manner. However, for ease of reference, only a limited number of sub-blocks are shown in full scan in FIG. 17. Progression from one 4×4 sub-block to the next also follows a diagonal backward scan across the entire TB 1700.

[0155] When using MTS, only the coefficients in the upper left 16x16 portion 1740 of the TB 1700 may be significant. The upper left 16x16 portion forms or falls within a threshold orthogonal position ((15,15) in this example) at which MTS can be applied. If the last significant coefficient is outside the threshold orthogonal position in either the X or Y coordinate, MTS cannot be applied. That is, if either the X or Y coordinate of the location of the last significant coefficient exceeds 15, MTS cannot be applied and a DCT-2 is applied (or the transform is skipped). The location of the last significant coefficient is expressed in orthogonal coordinates relative to the location of the DC coefficient in the TB 1700. For example, the location 1730 of the last significant coefficient is 15,15. The scan pattern 1710, which starts at location 1730 and progresses toward the DC coefficient, is zeroed out in the video encoder 114 when MTS is applied, resulting in scanning sub-blocks 1720 and 1721 (identified by shading) that are not used by the video decoder 134. Although the video decoder 134 needs to decode the residual coefficients of sub-blocks 1720 and 1721 because they are included in the scan, the decoded residual coefficients of sub-blocks 1720 and 1721 are not used when MTS is applied. At a minimum, the residual coefficients of sub-block 1720 may be required to be zero-valued in order for MTS to be applied, reducing the associated coding cost and preventing the bitstream from encoding significant residual coefficients of the sub-blocks when MTS is applied. That is, parsing the "mts_idx" syntax element may require not only that the last significant position be within portion 1740, but also that sub-blocks 1720 and 1721 contain only zero-valued residual coefficients.

[0156] 18 illustrates a scan pattern 1810 for a 32x32 TB 1800 using the described arrangement. Scan pattern 1810 groups 4x4 sub-blocks into several "collections," such as collection 1840.

[0157] In the context of this disclosure, in the context of a scan pattern, a collection provides a non-overlapping set of sub-blocks that (i) form an area or region of a size applicable to the MTS, or (ii) form an area or region that surrounds the area applicable to the MTS. The scan pattern traverses the transform block by progressing through several non-overlapping collections of sub-blocks of residual coefficients, progressing from the current collection to the next collection after completing the scan of the current collection.

[0158] In the example of Figure 18, each collection is a two-dimensional array of 4x4 subblocks with a width and height of up to four subblocks (collection option (i)). Collection 1840 corresponds to the region of potential significant coefficients when MTS is used, i.e., a 16x16 region of TB 1800. Scan pattern 1810 progresses from one collection to the next without re-entry; that is, once all residual coefficients in a collection have been scanned, scan pattern 1810 proceeds to the next collection. Scan 1810 effectively fully completes the scan pattern for the current collection before proceeding to scan the next collection. Collections are non-overlapping, and each residual coefficient position is scanned once, starting from the final position and progressing toward the DC (top-left) coefficient position.

[0159] Similar to scan pattern 1710, scan pattern 1810 also divides TU 1800 into 4x4 sub-blocks. Due to the monotonic progression from one collection to the next, once the scan reaches collection 1840 in the upper left, no further scanning of residual coefficients outside collection 1840 occurs. In particular, if the final position is within collection 1840, e.g., at final position 1830 at position 15,15, all residual coefficients outside collection 1840 are insignificant. Zeroing out residual coefficients outside 1840 is consistent with the zeroing out performed in video encoder 114 when MTS is used. Thus, video decoder 134 need only verify that the final position is within collection 1840 to enable parsing of the mts_idx syntax element (1122 when the CU belongs to a single coding tree, or 1226 when the CU belongs to the luma branch of another coding tree). The use of scan pattern 1810 eliminates the need to ensure that any residual coefficients outside collection 1840 are zero-valued. Coefficients outside collection 1840 are already known thanks to scan pattern 1810, which has a collection size aligned with the MTS transform coefficient domain. By dividing TB 1800 into a set of collections, each of which is the same size, scan pattern 1810 also enables reduced memory consumption compared to scan pattern 1710. A scan across TB 1800 can be constructed from a scan across one collection, resulting in memory savings. For TBs of size 16x32 and 32x16, two collections can be used using the same approach as for a 16x16 collection. For a 32x8 TB, the TB size allows for division into collections constrained to 16x8 size. Dividing a 32x8 TB into collections results in a scan pattern equivalent to a regular diagonal progression across an 8x2 array of 4x4 sub-blocks that make up the 32x8 TB. Therefore, in the area of ​​8x16 coefficients that are the subject of the 32x8TB MTS transform, the significant coefficient property is satisfied by checking that the final position is within the left half of 32x8TB.

[0160] FIG. 19 illustrates a TB 1900 of size 8×32. The TB 1900 can be divided into collections. In the example of FIG. 19, there is a collection, such as collection 1940, whose collection size is constrained to 8×16 due to TB size. Dividing the 8×32 TB 1900 into collections results in a different subblock order compared to the normal diagonal progression on a 2×8 array of 4×4 subblocks that make up the 8×32 TB (as shown, for example, in FIG. 18). Using an 8×16 collection size ensures that if the last significant coefficient position is within collection 1940, significant coefficients can only be in the MTS transform coefficient domain, e.g., the last significant position 1930 at 7,15.

[0161] The scan patterns in Figures 18 and 19 scan the residual coefficients of each sub-block diagonally backward. In the examples of Figures 18 and 19, the sub-blocks of each collection are scanned diagonally backward. Scanning between collections is done diagonally backward in Figures 18 and 19.

[0162] FIG. 20 illustrates an alternative scan order 2010 for a 32×32 TB 2000. The scan order (scan pattern) 2010 is divided into portions 2010a-2010f. Scan orders 2010a-2010e relate to option (ii) a collection of sub-blocks that form an area or region surrounding the area applicable to the MTS. Scan pattern 2010f relates to option (i) a collection that covers a region 2040 that forms the area applicable to the MTS. Scan orders 2010a-2010f are defined such that a backward diagonal progression from one sub-block to the next occurs across the TB 2000, excluding region 2040, which is then scanned using the backward diagonal scan. Region 2040 corresponds to the MTS transform coefficient region. Dividing TB 2000 into a scan on sub-blocks outside the MTS transform coefficient domain followed by a scan on sub-blocks within the MTS transform coefficient domain results in a progression on the sub-blocks as shown at 2010a, 2010b, 2010c, 2010d, 2010e, and 2010f. Scan pattern 2010 identifies two collections: a collection defined by 2010a through 2010e and a collection defined by region 2040 scanned by 2010f. The scan is performed in a manner that allows all sub-blocks bordering collection 2040 to be scanned before the lower right corner (2030) of collection 2040. Scan pattern 2010 scans the collection of sub-blocks formed using scans 2010a through 2010e. Once the collection covered by 2010a through 2010e is completed, scan pattern 2010 continues to the next collection 2040, which is scanned according to 2010f. To enable signaling of mts_idx, there is a property that checks that the position of the last significant coefficient, such as 2030, is within the region 2040, without also having to check that residual coefficients outside the region 2040 are zero-valued.

[0163] The scanning of the residual coefficients is done in a variation of the backward diagonal scan of Figure 20. The scan pattern scans the collection in a backward raster fashion in Figure 20. In a variation of the patterns of Figures 18 and 19, the collection may be scanned in backward raster order.

[0164] The scan patterns shown in Figures 18-20, i.e., 1810, 1910, and 2010a-f, substantially retain the progression from the highest frequency coefficient of the TB to the lowest frequency coefficient of the TB compared to scan pattern 1710 of Figure 17. Thus, the arrangements of video encoder 114 and video decoder 134 using scan patterns 1810, 1910, and 2010a-f achieve compression efficiency similar to that achieved when using scan pattern 1710, while being able to rely on the location of the last significant coefficient without the additional need to check for zero-valued residual coefficients outside the MTS transform coefficient region.

[0165] 13 illustrates a method 1300 for encoding frame data 113 into a bitstream 115, where the bitstream 115 includes one or more slices as a sequence of coding tree units. The method 1300 may be embodied by an apparatus such as an configured FPGA, ASIC, or ASSP. Furthermore, the method 1300 may be performed by the video encoder 114 under execution of the processor 205. As such, the method 1300 may be implemented as a module of software 233 stored on a computer-readable storage medium and / or in the memory 206.

[0166] The method 1300 begins with step 1310 of SPS / PPS encoding. In step 1310, the video encoder 114 encodes the SPS 1010 and PPS 1012 as a sequence of fixed-length and variable-length coding parameters into a bitstream 115. Parameters of the frame data 113, such as resolution and sample bit depth, are encoded. Bitstream parameters, such as flags indicating the use of specific coding tools, are also encoded. Picture parameter sets include parameters specifying how often the "delta QP" syntax element is present in the bitstream 113, the offset of the chroma QP relative to the luma QP, etc.

[0167] The method 1300 continues from step 1310 with a step 1320 of picture header encoding. In performing step 1320, the processor 205 encodes a picture header (e.g., 1015) into the bitstream 113, the picture header 1015 being applicable to all slices in the current frame. The picture header 1015 includes partitioning constraints that indicate the maximum allowable depth of binary, ternary, and quadtree partitioning, and may override similar constraints included as part of the SPS 1010.

[0168] The method 1300 continues from step 1320 with a slice header encoding step 1330. In step 1330, the entropy encoder 338 encodes the slice header 1118 into the bitstream 115.

[0169] From step 1330, method 1300 continues with step 1340, which involves dividing the slice into CTUs. In performing step 1340, video encoder 114 divides slice 116 into a sequence of CTUs. Slice boundaries are aligned with CTU boundaries, and CTUs within the slice are ordered according to CTU scan order, typically raster scan order. The division of the slice into CTUs establishes the order in which portions of frame data 113 should be processed by video encoder 113 when encoding each current slice.

[0170] From step 1340, method 1300 continues with coding tree determination step 1350. In step 1350, video encoder 114 determines a coding tree for the currently selected CTU in the slice. Method 1300 starts with the first CTU in slice 0116 in the first invocation of step 1350 and progresses to subsequent CTUs in slice 0116 in subsequent invocations. In determining the coding tree for the CTU, various combinations of quadtree, binary, and ternary partitioning are generated and tested by block partitioner 310.

[0171] From step 1350, method 1300 continues with coding unit determination step 1360. In step 1360, video encoder 114 performs, using known methods, to determine encodings for CUs resulting from the various coding trees under evaluation. Determining the encoding includes determining a prediction mode (e.g., intra prediction 387 with a specific mode or inter prediction with motion vectors) and a primary transform type 389. If primary transform type 389 is determined to be DCT-2 and all quantized primary transform coefficients that do not undergo a forward secondary transform are insignificant, a secondary transform index 388 is determined and may indicate application of a secondary transform (e.g., encoding as 1120, 1224, or 1254). Otherwise, secondary transform index 388 indicates bypassing the secondary transform. Furthermore, a transform skip flag 390 is determined for each TB in the CU to indicate whether to apply a primary transform (and optionally a secondary transform) or to bypass the transform entirely (e.g., 1126 / 1130 / 1134 or 1232 / 1262 / 1266). For the luma channel, the type of primary transform is determined to be one of DCT-2, transform skip, or MTS options, and for the chroma channels, DCT-2 or transform skip are available transform types. Determining the encoding may also include determining a quantization parameter that can change the QP, i.e., a “delta QP” syntax element is coded into the bitstream 115. When determining individual coding units, the optimal coding tree is also jointly determined. If the coding units in the shared coding tree are coded using intra prediction, the luma intra prediction mode and chroma intra prediction are determined in step 1360. If a coding unit in a separate coding tree is to be coded using intra prediction, then either the luma intra prediction mode or the chroma intra prediction mode is determined in step 1360 depending on whether the branch of the coding tree is luma or chroma, respectively.

[0172] The coding unit determination step 1360 may inhibit the testing and application of a secondary transform when there are no "AC" residual coefficients in the primary domain residual resulting from the application of the DCT-2 primary transform by the forward primary transform module 326. AC residual coefficients are residual coefficients at locations other than the top-left location of the transform block. The inhibition of testing a secondary transform when only a DC primary coefficient is present extends to the block to which the secondary transform index 388 is applied, i.e., the Y, Cb, and Cr of the shared tree (which is the Y channel only if the Cb, Cr block is two samples wide or high). Regardless of whether the coding unit is for a shared tree or a separate tree, if there is at least one significant AC primary coefficient, the video encoder 114 tests for the selection of a non-zero secondary transform index value 388 (i.e., for the application of a secondary transform).

[0173] Method 1300 continues from step 1360 with coding unit encoding step 1370. In step 1370, video encoder 114 encodes the determined coding unit of step 1360 into bitstream 115. An example of how a coding unit is encoded is described in more detail with reference to FIG.

[0174] Method 1300 continues from step 1370 to a last coding unit test step 1380. In step 1380, processor 205 tests whether the current coding unit is the last coding unit of the CTU. If not ("NO" at step 1380), control within processor 205 returns to coding unit determination step 1360. Otherwise, if the current coding unit is the last coding unit ("YES" at step 1380), control within processor 205 proceeds to a last CTU test step 1390.

[0175] In last CTU test step 1390, processor 205 tests whether the current CTU is the last CTU in slice 1316. If the current CTU is not the last CTU in the slice (step 1390, 1016 "NO"), control in processor 205 returns to decision coding tree step 1350. Otherwise, if the current CTU is the last (step 1390, "YES"), control in processor 205 proceeds to last slice test step 13100.

[0176] In last slice test step 13100, processor 205 tests whether the current slice being coded is the last slice in the frame. If the current slice is not the last slice ("NO" in step 13100), control in processor 205 returns to slice header coding step 1330. Otherwise, if the current slice is the last slice and all slices have been coded ("YES" in step 13100), method 1300 ends.

[0177] Figure 14 illustrates a method 1400 for encoding coding units into bitstream 115, corresponding to step 1370 of Figure 13. Method 1400 may be embodied by an apparatus such as an configured FPGA, ASIC, or ASSP. Furthermore, method 1400 may be performed by video encoder 114 under execution by processor 205. As such, method 1400 may be stored on a computer-readable storage medium and / or in memory 206 as a module of software 233.

[0178] Method 1400 results in improved compression efficiency by encoding secondary transform index 1254 only if it can be applied to the chroma TB of TU 1260 and encoding secondary transform index 1120 only if it can be applied to any of the TBs of TU 1124. When a shared coding tree is used, method 1400 is invoked for each CU in the coding tree, e.g., CU 1114 in FIG. 11, to encode the Y, Cb, and Cr color channels. When separate coding trees are used, method 1400 is first invoked for each CU in luma branch 1214a, e.g., 1220, and method 1400 is also invoked for each chroma CU in chroma branch 1214b, e.g., 1250.

[0179] The method 1400 begins with a prediction block generation step 1410. In step 1410, the video encoder 114 generates a prediction block 320 according to the prediction mode of the CU determined in step 1360, e.g., the intra prediction mode 387. The entropy encoder 338 encodes the intra prediction mode 387 for the coding unit determined in step 1360 into the bitstream 115. A "pred_mode" syntax element is encoded to distinguish between the use of intra prediction, inter prediction, or other prediction modes for the coding unit. If intra prediction is used for the coding unit, a luma intra prediction mode is encoded if a luma PB is applicable to the CU, and a chroma intra prediction mode is encoded if a chroma PB is applicable to the CU. That is, for an intra-predicted CU belonging to a shared tree, such as the CU 1114, the prediction mode 1116 includes a luma intra prediction mode and a chroma intra prediction mode. For an intra-predicted CU that belongs to the luma branch of a separate coding tree, such as CU 1220, prediction mode 1221 includes a luma intra prediction mode. For an intra-predicted CU that belongs to the chroma branch of a separate coding tree, such as CU 1250, prediction mode 1251 includes a chroma intra prediction mode. Primary transform type 389 is coded to select from using DCT-2 horizontally and vertically, transform skip horizontally and vertically, or a combination of DCT-8 and DST-7 horizontally and vertically for the luma TB of the coding unit.

[0180] The method 1400 continues from step 1410 with a residual determination step 1420. The prediction block 320 is subtracted from the corresponding block of frame data 312 by a difference module 322 to generate a difference 324.

[0181] Method 1400 continues from step 1420 to a residual transform step 1430. In the residual transform step 1430, video encoder 114, under the execution of processor 205, either bypasses the primary and secondary transforms on the residual of step 1420 or performs a transform according to primary transform type 389 and secondary transform index 388 for each TB of the CU. The transform of difference 324 may be performed or bypassed according to transform skip flag 390, and if transformed, a secondary transform may also be applied as determined in step 1350 to generate residual samples 350, as described with reference to FIG. 3. After operation of quantification module 334, residual coefficients 336 are available.

[0182] The method 1400 continues from step 1430 with step 1440 of luma transform skip flag encoding. In step 1440, the entropy encoder 338 encodes the context-coded transform skip flag 390 into the bitstream 115 to indicate either that the residual of the luma TB is transformed according to a primary transform and possibly a secondary transform, or that the primary and secondary transforms are bypassed. Step 1440 is performed when the CU includes a luma TB, i.e., in the luma branch of the shared coding tree (encoding 1126) or the dual tree (encoding 1232).

[0183] From step 1440, the method 1400 continues with step 1450 of luma residual coding. In step 1450, the entropy encoder 338 encodes the residual coefficients 336 for the luma TB into the bitstream 115. Step 1450 operates to select an appropriate scan pattern based on the size of the coding unit. Examples of scan patterns are described with reference to Figure 17 (traditional scan pattern) and Figures 18-20 (additional scan patterns used to determine the MTS flag). In the embodiments described herein, the scan patterns associated with the examples of Figures 18-20 are used. The residual coefficients 336 are typically scanned into a list according to a backward diagonal scan pattern with 4x4 sub-blocks. For TBs with a width or height greater than 16 samples, the scan pattern is as described with reference to Figures 18, 19, and 20. The location of the first non-zero residual coefficient in the list (i.e., 1140) is encoded into the bitstream 115 as a Cartesian coordinate relative to the top-left coefficient of the transform block. The remaining residual coefficients are coded as residual coefficients 1144, starting from the coefficient in the final position to the DC (top-left) residual coefficient. Step 1450 is performed if the CU contains a luma TB, i.e., a shared coding tree (coding 1128), or if the CU belongs to the luma branch of a dual tree (coding 1234).

[0184] Method 1400 continues from step 1450 with step 1460 of chroma transform skip flag encoding. In step 1460, entropy encoder 338 encodes into bitstream 115 two more context coding transform skip flags 390, one for each chroma TB, that indicate whether the corresponding TB undergoes a DCT-2 transform and optionally a secondary transform, or whether the transform is bypassed. Step 1460 is performed if the CU contains chroma TBs, i.e., in the shared coding tree (coding 1130 and 1134) or the chroma branches of the dual tree (coding 1262 and 1266).

[0185] Method 1400 continues from step 1460 with step 1470 of chroma residual coding. In step 1470, the entropy encoder 338 encodes the residual coefficients of the chroma TB into the bitstream 115, as described with reference to step 1450. Step 1460 is performed when the CU includes a chroma TB, i.e., in the shared coding tree (coding 1132 and 1136) or the chroma branch of the dual tree (coding 1264 and 1268). For chroma TBs with a width or height greater than 16 samples, the scan patterns are as described with reference to Figures 18, 19, and 20. Using the scan patterns of Figures 18-20 for the luma and chroma TBs avoids the need to define different scan patterns between luma and chroma for TBs of the same size.

[0186] Method 1400 continues from step 1470 to step 1480 of the LFNST signaling test. In step 1480, processor 205 determines whether a secondary transform can be applied to any TB of the CU. If all of the TBs of the CU use transform skipping, there is no need to code the secondary transform index 388 (“NO” in step 1480), and method 1400 proceeds to step 14100 of the MTS signaling test. In the case of a shared coding tree, for example, the luma TB and each of the two chroma TBs are transform skipped for returning “NO” in step 1480. In the case of separate coding trees, a luma TB in the luma branch of the coding tree is transform skipped for step 1480 for returning “NO” to the call for luma and chroma, respectively, or two chroma TBs in the chroma branch of the coding tree are both transform skipped. For a secondary transform to be performed, the TB in question only needs to contain significant residual coefficients at the position of the TB targeted for the secondary transform. That is, all other residual coefficients must be zero, and this condition is achieved if the final position of the TB is within 806, 824, 842, or 862 for the TB sizes shown in Figures 8A-8D. If the final position of any TB in the CU is outside 806, 824, 842, or 862 for the considered TB size, no secondary transformation is performed ("NO" at step 1480) and method 1400 proceeds to step 14100 of the MTS signaling test.

[0187] In the case of chroma TBs, a width or height of 2 may occur. A TB with a width or height of 2 is not subject to secondary transformation because there is no kernel defined for a TB of that size ("NO" in step 1480), and method 1400 proceeds to step 14100 of the MTS signaling test. An additional condition for performing secondary transformation is that at least AC residual coefficients exist in the applicable TB. That is, if significant residual coefficients exist only in the DC (top left) position of each applicable TB, secondary transformation is not performed ("NO" in step 1480), and method 1400 proceeds to step 14100 of the MTS signaling test. If at least one TB of the CU is subject to a primary transform (the transform skip flag indicates no skipping for at least one TB of the CU), the sampling position constraints for the TBs subject to the primary transform are satisfied, and at least one AC coefficient is included in one or more of the TBs subject to the primary transform ("YES" in step 1480), control in the processor 205 proceeds to LFNST index encoding step 1490. In LFNST index encoding step 1490, the entropy encoder 338 encodes a truncated unary codeword indicating three possible choices for applying a secondary transform. The choices are zero (not applied), one (the first kernel in the candidate set is applied), and two (the second kernel in the candidate set is applied). The codeword uses a maximum of two bins, and each bin is context coded. Due to the test performed in step 1480, step 1490 is only executed when a secondary transform can be applied, i.e., for the non-zero indices to be coded. Step 1490 encodes, for example, 1120 or 1224 or 1225.

[0188] In effect, the operations of steps 1480 and 1490 allow the secondary transform index 1254 for chroma in the separate tree structure to be coded only if the secondary transform can be applied to the chroma TB of TU 1260. In a shared tree structure, steps 1480 and 1490 operate to code the secondary transform index 1120 only if the secondary transform can be applied to any of the TBs of TU 1124. In excluding related secondary transform indexes (such as 1254 and 1120), method 1400 operates to improve coding efficiency. In particular, in the case of a shared or dual tree, unnecessary flags are avoided, thereby reducing the number of required bits and improving coding efficiency. In the case of separate trees, the secondary transform is not necessarily suppressed for chroma if the corresponding luma transform block is transform-skipped.

[0189] Method 1400 proceeds from step 1490 to step 14100 of the MTS signaling test.

[0190] In MTS signaling step 14100, the video encoder 114 determines whether an MTS index needs to be coded into the bitstream 115. If the use of a DCT-2 transform was selected in step 1360, the last significant coefficient position may be anywhere within the top-left 32x32 region of the TB. If the last significant coefficient position is outside the top-left 16x16 region of the TB and the scan of Figures 18 and 19 is used (rather than the scan pattern of Figure 17), there is no need to explicitly signal mts_idx in the bitstream. Because the use of MTS does not generate a last significant coefficient outside the top-left 16x16 region, in this case, the signal mts_idx is not needed in the bitstream. If step 14100 returns "NO," the method 1400 ends with the use of a DCT-2 transform implied by the location of the last significant coefficient.

[0191] The non-DCT-2 selection of the primary transform type is only available if the width and height of the TB are less than or equal to 32. Therefore, for TBs with width or height greater than 32, step 14100 returns "NO" and method 1400 ends at step 14100. The non-DCT-2 selection is also only available if no secondary transform is applied; therefore, if step 1360 determines that secondary transform type 388 is non-zero, step 14100 returns "NO" and method 1400 ends at step 14100.

[0192] 18 and 19, the location of the last significant coefficient within the top left 16x16 region of the TB can result from either the application of a DCT-2 linear transform or an MTS combination of DCT-7 and / or DCT-8, so explicit signaling of mts_idx is required to encode the selection made in step 1360. Thus, when the last significant coefficient location is within the top left 16x16 region of the TB, step 14100 returns "YES" and method 1400 proceeds to step 14110 for MTS index encoding.

[0193] In MTS index encoding step 14110, the entropy encoder 338 encodes the truncated unary bin string representing the linear transform type 389. Step 14110 may, for example, encode 1122 or 1226. The method 1400 ends with the execution of step 14110.

[0194] 15 illustrates a method 1500 for decoding a bitstream 133 to generate frame data 135, where the bitstream 133 includes one or more slices as a sequence of coding tree units. The method 1500 may be embodied by an apparatus such as an configured FPGA, ASIC, or ASSP. Furthermore, the method 1500 may be performed by the video decoder 134 under execution by the processor 205. As such, the method 1500 may be stored as one or more modules of the software 233 on a computer-readable storage medium and / or in the memory 206.

[0195] The method 1500 begins with step 1510 of SPS / PPS decoding. In step 1510, the video decoder 314 decodes the SPS 1010 and PPS 1012 from the bitstream 133 as a sequence of fixed-length and variable-length coding parameters. Parameters of the frame data 113, such as resolution and sample bit depth, are decoded. Bitstream parameters, such as flags indicating the use of specific coding tools, are also decoded. Default partition constraints signal the maximum allowed depth of binary, ternary, and quadtree partitioning and may also be decoded by the video decoder 134 as part of the SPS 1010.

[0196] Method 1500 continues from step 1510 with a step 1520 of picture header decoding. In performing step 1520, processor 205 decodes from bitstream 113 a picture header 1015 applicable to all slices in the current frame. The picture parameter set includes parameters specifying how often a "delta QP" syntax element is present in bitstream 313, the offset of the chroma QP relative to the luma QP, etc. Optional overridden partition constraints signal the maximum allowed depth of binary, ternary, and quadtree partitioning and may also be decoded by video decoder 134 as part of the picture header 1015.

[0197] The method 1500 continues from step 1520 with a slice header decoding step 1530. In step 1530, the entropy decoder decodes 420 the slice header 0118 from the bitstream 133.

[0198] From step 1530, method 1500 continues with step 1540, which involves dividing the slice into CTUs. In performing step 1540, video encoder 114 divides slice 1016 into a sequence of CTUs. Slice boundaries are aligned with CTU boundaries, and CTUs within the slice are ordered according to CTU scan order, typically raster scan order. The division of the slice into CTUs establishes which portions of frame data 133 should be processed by video encoder 313 when decoding the current slice.

[0199] Method 1500 continues from step 1540 with coding tree decoding step 1550. In step 1550, video decoder 314 decodes the coding tree of the currently selected CTU in the slice. Method 1500 starts with the first CTU in the slice 1016 in the first invocation of step 1550 and progresses to subsequent CTUs in the slice 1016 in subsequent invocations. In decoding the coding tree of the CTU, a flag indicating the combination of quadtree, binary, and ternary partitioning determined in step 1350 in video encoder 114 is decoded.

[0200] Method 1500 continues from step 1550 with coding unit decoding step 1570. In step 1570, video decoder 314 decodes the determined coding unit of step 1560 from bitstream 133. An example of how a coding unit is decoded is described in more detail with reference to FIG.

[0201] Method 1500 continues from step 1570 to a last coding unit test step 1580. In step 1580, processor 205 tests whether the current coding unit is the last coding unit of the CTU. If not ("NO" at step 1580), control within processor 205 returns to coding unit decoding step 1560. Otherwise, if the current coding unit is the last coding unit ("YES" at step 1580), control within processor 205 proceeds to a last CTU test step 1590.

[0202] In last CTU test step 1590, processor 205 tests whether the current CTU is the last CTU of slice 1016. If it is not the last CTU of slice 1016 ("NO" in step 1590), control in processor 205 returns to coding tree decoding step 1550. Otherwise, if the current CTU is the last ("YES" in step 1590), control in processor 205 proceeds to last slice test step 15100.

[0203] In last slice test step 15100, processor 205 tests whether the current slice being decoded is the last slice in the frame. If the current slice is not the last slice ("NO" in step 15100), control in processor 205 returns to slice header decoding step 1530. Otherwise, if the current slice is the last slice and all slices have been decoded ("YES" in step 15100), method 1500 ends.

[0204] Figure 16 illustrates a method 1600 for decoding coding units from bitstream 133, corresponding to step 1570 of Figure 15. Method 1600 may be embodied by an apparatus such as an configured FPGA, ASIC, or ASSP. Furthermore, method 1600 may be performed by video decoder 314 under execution by processor 205. As such, method 1600 may be stored on a computer-readable storage medium and / or as one or more modules of software 233 in memory 206.

[0205] If a shared coding tree is used, method 1600 is called for each CU in the coding tree, e.g., CU 1114 in Figure 11, and the Y, Cb, and Cr color channels are coded in a single call. If separate coding trees are used, method 1600 is first called for each CU in the luma branch 1214a, e.g., 1220, and method 1600 is also called separately for each chroma CU in the chroma branch 1214b, e.g., 1250.

[0206] The method 1600 begins with step 1610 of luma transform skip flag decoding. In step 1610, the entropy decoder 420 decodes the context-coded transform skip flag 478 (e.g., coded in the bitstream as 1126 in FIG. 11 or 1232 in FIG. 12) from the bitstream 133. The skip flag indicates whether a transform is applied to the luma TB. The transform skip flag 478 indicates that the residual for the luma TB is transformed according to (i) a primary transform, (ii) a primary transform and a secondary transform, or (iii) the primary transform and the secondary transform are bypassed. Step 1610 is performed when the CU includes the luma TB in the shared coding tree (e.g., decoding 1126). Step 1610 is performed when the CU belongs to the luma branch of the dual tree (decoding 1232) of a separate coding tree CTU.

[0207] Method 1600 continues from step 1610 with step 1620 of luma residual decoding. In step 1620, entropy decoder 420 decodes 424 residual coefficients for the luma TB from bitstream 115. The residual coefficients 424 are combined into the TB by applying a scan to the list of decoded residual coefficients. Step 1620 operates to select an appropriate scan pattern based on the size of the coding unit. Examples of scan patterns are described in relation to Figure 17 (traditional scan pattern) and Figures 18-20 (additional scan patterns useful for determining the MTS flag). In the examples described herein, a scan pattern based on the pattern described in relation to Figures 18-20 is used. This scan is typically a backward diagonal scan pattern using 4x4 sub-blocks, as defined with reference to Figures 18 and 19. The position of the first non-zero residual coefficient in the list (i.e., 1140) is decoded from bitstream 133 as a Cartesian coordinate relative to the top-left coefficient of the transform block. The remaining residual coefficients are decoded as residual coefficients 1144 in order from the coefficient in the last position to the DC (top left) residual coefficient.

[0208] For each subblock other than the top-left subblock of the TB and the subblock containing the last significant residual coefficient, a "coded subblock flag" indicating that the respective subblock has at least one significant residual coefficient is decoded. If the coded subblock flag indicates the presence of at least one significant residual coefficient within the subblock, a "significance map" (a set of flags) is decoded to indicate the significance of each residual coefficient within the subblock. If the decoded coded subblock flag indicates that the subblock contains at least one significant residual coefficient and the scan reaches the last scan position of the subblock without encountering a significant residual coefficient, the residual coefficient at the last scan position of the subblock is presumed to be significant. The coded subblock flags and significance map (each flag is named "sig_coeff_flag") are coded using context-coded bins. For each significant residual coefficient within the subblock, an "abs_level_gtx_flag" is decoded to indicate whether the magnitude of the corresponding residual coefficient is greater than 1. For each residual coefficient in the sub-block with a magnitude greater than 1, the 'par_level_flag' and 'abs_level2_gtx_flag' are decoded to further determine the magnitude of the residual coefficient according to equation (1). AbsLevelPass1 = sig_coeff_flag + par_level_flag + abs_level_gtx_flag + 2×abs_level_gtx_flag2 (1)

[0209] The abs_level_gtx_flag and abs_level_gtx_flag2 syntax elements are coded using context-coded bins. For each residual coefficient with abs_level_gtx_flag2 equal to 1, the bypass-coded syntax element "abs_remainder" is decoded using Rice-Golon coding. The decoded magnitude of the residual coefficient is determined as follows: AbsLevel = AbsLevelPass1 + 2 × abs_remainder. To obtain the value of the residual coefficient from the magnitude of the residual coefficient, the sign bit of each significant residual coefficient is decoded. The Cartesian coordinates of each subblock in the scan pattern can be derived from the scan pattern by adjusting (right-shifting) the Cartesian coordinates of the X and Y residual coefficients by the log2 of the width and height of the subblock, respectively. For luma TB, the subblock size is always 4 × 4, and X and Y are right-shifted by 2 bits. 18-20 can also be applied to the chroma TB to avoid storing different scan patterns for blocks of the same size but different color channels. Step 1620 is performed when the CU includes a luma TB, i.e., in the shared coding tree (decoding 1128) or for a call to the luma branch of the dual tree (e.g., decoding 1234).

[0210] Method 1600 continues from step 1620 to step 1630 of decoding chroma transform skip flags. In step 1630, the entropy decoder decodes 420 context-coded flags from bitstream 1 for each chroma TB. (For example, context-coded flags may be coded as 1130 and 1134 in FIG. 11 or 1262 and 1266 in FIG. 12.) At least one flag is decoded for each chroma TB. The flag decoded in step 1630 indicates whether a transform is applied to the corresponding chroma TB, in particular whether a DCT-2 transform and optionally a secondary transform are applied to the corresponding chroma TB, or whether all transforms for the corresponding chroma TB are bypassed. Step 1630 is performed if the CU includes a chroma TB, i.e., if the CU belongs to a shared coding tree (decodes 1130 and 1134) or a chroma branch of a dual tree (decodes 1262 and 1266).

[0211] Method 1600 continues from step 1630 with step 1640 of chroma residual decoding, in which entropy decoder 420 decodes residual coefficients of the chroma TB from bitstream 133. Step 1640 operates in a similar manner as described with reference to step 1620 and according to the scan pattern defined in Figures 18 and 19. Step 1640 is performed when a CU includes a chroma TB, i.e., when the CU belongs to a shared coding tree (decodes 1132 and 1136) or a chroma branch of a dual tree (decodes 1264 and 1268).

[0212] Method 1600 continues from step 1640 to step 1650 of LFNST signaling test. In step 1650, processor 205 determines whether a secondary transform is applicable to any TB of the CU. The luma transform skip flag may have a different value than the chroma transform skip flag. If all of the TBs of the CU use transform skip, the secondary transform is not applicable, there is no need to code a secondary transform index (“NO” in step 1650), and method 1600 proceeds to step 1660 of LFNST index determination. For example, in the case of a shared coding tree, the luma TB and each of the two chroma TBs are transform-skipped to return “NO” in step 1650. For a CU that belongs to the luma branch of a separate coding tree (e.g., 1220), step 1650 returns “NO” if the luma TB is transform-skipped. For a CU (e.g., 1250) belonging to a chroma branch of a separate coding tree, step 1650 returns "NO" if both chroma TBs are to be transformed and skipped. For a CU (e.g., 1250) belonging to a chroma branch of a separate coding tree and having a width or height of less than four samples, step 1650 returns "NO." To perform a secondary transform, the corresponding TB only needs to contain significant residual coefficients at the TB position targeted for the secondary transform. That is, all other residual coefficients must be zero. This condition is achieved when the final position of the TB is within 806, 824, 842, or 862 for the TB sizes shown in Figures 8A-8D. If the final position of any TB in the CU is outside 806, 824, 842, or 862 for the considered TB size, the secondary transform is not performed ("NO" in step 1650), and method 1600 proceeds to step 1660 for LFNST index determination. For a chroma TB, two widths or heights are possible. A TB with a width or height of 2 is not subject to secondary transformation because there is no kernel defined for a TB of that size. An additional condition for secondary transformation is that there is at least one AC residual coefficient in the TB.That is, if significant residual coefficients are only in the DC (top-left) position for each TB, no secondary transform is performed ("NO" at step 1650), and method 1600 proceeds to LFNST index determination step 1660. The constraints on the location of the last significant coefficient and the presence of non-DC residual coefficients only apply to TBs of applicable size, i.e., TBs with width and height greater than two samples. Provided that at least one applicable TB has been transformed, the final position constraints are satisfied, and the non-DC coefficient requirement is met ("YES" at step 1650), control within processor 205 proceeds to LFNST index decoding step 1670.

[0213] Step 1660 of determining an LFNST index is performed if a secondary transform cannot be applied to any of the TBs associated with a CU. In step 1660, processor 205 determines that the secondary transform index has a value of zero, indicating that no secondary transform is applied. Control in processor 205 proceeds from step 1660 to step 1672 of MTS signaling.

[0214] In step 1670 of LFNST index decoding, the entropy decoder 420 decodes the truncated unary codeword as a secondary transform index 474, which indicates three possible choices for applying the secondary transform. The choices are 0 (not applied), 1 (the first kernel in the candidate set is applied), and 2 (the second kernel in the candidate set is applied). The codeword uses at most two bins, and each bin is context coded. Due to the test performed in step 1650, step 1670 is executed only if the secondary transform can be applied, i.e., a non-zero index can be decoded. When method 1600 is invoked as part of a shared coding tree, step 1670 decodes bitstreams 133 to 1120. When method 1600 is invoked as part of the luma branch of a separate coding tree, step 1670 decodes bitstreams 133 to 1224. When step 1670 is invoked as part of the chroma branch of a separate coding tree, step 1670 decodes bitstreams 133 to 1254. Control within processor 205 proceeds from step 1670 to step 1672 of MTS signaling.

[0215] Steps 1650, 1660, and 1670 operate to determine the LFNST index, i.e., 474. The LFNST index is decoded from the video bitstream (e.g., decode 1120, 1224, or 1254) if at least one of the luma transform skip flag and the chroma transform skip flag applicable to the CU indicates that the transform of the respective transform block is not skipped ("YES" in step 1650, and perform step 1670). The LFNST index is determined to indicate that no secondary transform is applied if all of the luma transform skip flags and the chroma transform skip flags applicable to the CU indicate that the transform of the respective transform block is skipped ("NO" in step 1650, and perform step 1660). In the case of a shared tree, the luma skip value and the chroma skip value and the LFNST index can be different. For example, the decoded LFNST index for a chroma transform block can be based on the decoded chroma skip flag, even if, for example, in a collocated block, the decoded luma transform skip flag indicates that the transform for the luma block is skipped. Encoding steps 1480 and 1490 operate in a similar manner.

[0216] In step 1672 of MTS signaling, video decoder 114 determines whether it needs to decode an MTS index from bitstream 133. If the use of a DCT-2 transform was selected in step 1360 when encoding the bitstream, the last significant coefficient location can be anywhere within the top-left 32x32 region of the TB. If the location of the last significant coefficient decoded in step 1620 is outside the top-left 16x16 region of the TB and the scanning of Figures 18 and 19 is used, there is no need to explicitly decode mts_idx because using a linear transform other than DCT-2 would not produce a last significant coefficient outside this region. If step 1672 returns "NO," method 1600 proceeds from step 1672 to step 1674 of MTS index determination. Non-DCT2 linear transforms are available only when the TB width and height are less than or equal to 32. Thus, for TBs with width or height greater than 32, step 1672 returns "NO" and method 1600 proceeds to step 1674 of MTS index determination.

[0217] A non-DCT-2 primary transform is available only if secondary transform type 474 indicates bypassing application of the secondary transform kernel; accordingly, if secondary transform type 474 has a non-zero value, method 1600 proceeds from step 1672 to step 1674. When using the scans of Figures 18 and 19, the presence of the last significant coefficient position within the top-left 16x16 region of the TB can result from either application of a DCT-2 primary transform or an MTS combination of DCT-7 and / or DCT-8, so explicit signaling of mts_idx is required to encode the selection made in step 1360. Thus, when the last significant coefficient position is within the top-left 16x16 region of the TB, step 1672 returns "YES," and method 1600 proceeds to step 1676 of MTS index decoding.

[0218] In MTS index determination step 1674, the video decoder 134 determines to use DCT-2 as the primary transform. Primary transform type 476 is set to zero. Method 1400 proceeds from step 1674 to a transform residual step 1680.

[0219] In MTS index decoding step 1676, entropy decoder 420 decodes the truncated unary bin string from bitstream 133 to determine primary transform type 476. The truncated string is in the bitstream, for example, as 1122 in Figure 11 or 1226 in Figure 12. Method 1400 proceeds from step 1676 to residual transform step 1680.

[0220] Steps 1670, 1672, and 1674 operate to determine an MTS index for the coding unit. The MTS index is decoded from the video bitstream if the last significant coefficient is at or within the threshold coordinate (15, 15) (“YES” in steps 1672 and 1676). The MTS index is determined to indicate that MTS is not to be applied if the last significant coefficient is outside the threshold coordinate (“NO” in steps 1672 and 1674). Encoding steps 14100 and 14110 operate in a similar manner.

[0221] In an alternative arrangement of the video encoder 114 and the video decoder 134, a chroma TB of appropriate size (no MTS is applied to the chroma TB) is scanned according to the scan pattern as described with reference to Figure 17, and the luma TB utilizes scanning according to Figures 18 and 19, with the DST-7 / DCT-8 combination applied only to the luma TB.

[0222] In residual transform step 1680, the video decoder 314, under the execution of processor 205, either bypasses the inverse primary and inverse secondary transforms on the residual of step 1420 or performs an inverse transform according to the primary transform type 476 and the secondary transform index 474. The transform is performed for each TB of the CU according to the decoded transform skip flag 478 for each TB of the CU, as described with reference to FIG. 4. The primary transform type 476 selects whether to use DCT-2 horizontally and vertically or a combination of DCT-8 and DST-7 horizontally and vertically for the luma TB of the coding unit. Essentially, step 1680 transforms the luma transform block of the CU according to the decoded luma transform skip flag, primary transform type 476, and the secondary transform index determined by the operations of steps 1610 and 1650 through 1670, and decodes the coding unit. Step 1680 may also decode the coding unit by transforming the chroma transform blocks of the CU according to the respective decoded chroma transform skip flags and secondary transform indexes determined by the operations of steps 1630 and 1650 to 1670. For TBs belonging to chroma channels (e.g., 1132 and 1136 in the case of a shared coding tree, or 1264 and 1268 in the case of a separate coding tree), secondary transforms are performed only when the width and height of the TB are four samples or greater because there are no available secondary transform kernels for TBs with a width or height less than four samples. For TBs belonging to chroma channels, the VVC standard imposes a partitioning operation restriction that prohibits intra-predicted CUs with TB sizes of 2x2, 2x4, and 4x2 due to the difficulty of processing TBs of such small sizes at the block throughput speeds required to support video formats such as UHD and 8K. Furthermore, there is a restriction that prohibits intra-predicted CUs with TBs of width 2 due to the difficulty of memory access for the on-chip memory typically used to generate reconstructed samples as part of the intra prediction operation. Therefore, the chroma TB sizes (chroma sample units) without secondary transformation are shown in Table 1. [Table 1]

[0223] As described herein, different scan patterns can be used in encoding and decoding. Step 1680 transforms the transform blocks of the CU according to the MTS index and decodes the coding unit.

[0224] Method 1600 continues from step 1680 to step 1690 of generating a prediction block. In step 1690, video decoder 134 generates a prediction block according to the prediction mode of the CU as determined in step 1360 and decoded from bitstream 113 by entropy decoder 420 452. Entropy decoder 420 decodes the prediction mode for the coding unit as determined in step 1360 from bitstream 133. To distinguish between the use of intra-prediction, inter-prediction, or other prediction modes for a coding unit, the "pred_mode" syntax element is decoded. If intra-prediction is used for a coding unit, a luma intra prediction mode is decoded if a luma PB is applicable to the CU, and a chroma intra prediction mode is decoded if a chroma PB is applicable to the CU.

[0225] Method 1600 continues from step 1690 with coding unit reconstruction step 16100. In step 16100, the prediction block is added 452 to residual samples 424 for each color channel of the CU to generate reconstructed samples 456. Additional in-loop filtering steps, such as deblocking, may be applied to the reconstructed samples 456 before being output as frame data 135. Method 1600 ends with the execution of step 16100.

[0226] As mentioned above, in the case of separate coding trees, method 1600 is first invoked for each CU in luma branch 1214a, e.g., 1220, and method 1600 is also invoked separately for each chroma CU in chroma branch 1214b, e.g., 1250. The invocation of method 1600 for chroma determines the LFNST index 1254 in steps 1650 through 1670 with respect to whether all of the chroma transform skip flags of CU 1250 are set. Similarly, in the invocation of method 1600 for luma, the luma LFNST index 1224 is determined in steps 1650 through 1670 with respect to the luma transform skip flag of CU 1220 only.

[0227] The scan patterns shown in Figures 18-20, i.e., 1810, 1910, and 2010a-f, implemented in steps 1450 and 1620, substantially retain the characteristic of progressing from the highest-frequency coefficient of the TB to the lowest-frequency coefficient of the TB, compared to scan pattern 1710 of Figure 17. Therefore, the arrangement of video encoder 114 and video decoder 134 using scan patterns 1810, 1910, and 2010a-f achieves compression efficiency similar to that achieved when using scan pattern 1710, while allowing them to rely on the last significant coefficient location without the additional need to check for zero-valued residual coefficients outside the MTS transform coefficient region. The final location used in the scan patterns of Figures 18-20 allows the MTS to be used only if all significant coefficients are in the appropriate upper-left region, such as the upper-left 16x16 region. The burden on decoder 134 to check flags outside the appropriate region, e.g., outside the TB 16x16 coefficient region, to ensure that no further insignificant coefficients exist, is eliminated. The operation in the decoder does not require any specific modifications to implement MTS. Furthermore, as mentioned above, the use of the scan patterns of Figures 18 and 19, i.e., for transform blocks of sizes 16x32, 32x16, and 32x32, can be replicated from the 16x16 scan, thereby reducing memory requirements. [Industrial Applicability]

[0228] The method has applications in the computer and data processing industry, particularly in digital signal processing for encoding and decoding video and image signals, achieving high compression efficiency.

[0229] Some arrangements described herein improve compression efficiency by signaling secondary transform indexes when available choices include at least one option other than bypassing the secondary transform. This improvement is achieved both when a CTU is divided into CUs spanning all color channels (a "shared coding tree" case) and when a CTU is divided into a set of luma CUs and a set of chroma CUs (a "separate coding tree" case). In the case of a separate tree, redundant signaling of secondary transform indexes is avoided when they cannot be used. In the shared tree, LFNST indexes can be signaled even if luma uses transform skip in the primary case of chroma DCT-2. Other arrangements maintain compression efficiency while allowing MTS index signaling to rely on the last significant coefficient position without the additional need to check for zero-valued residual coefficients outside the MTS transform coefficient region of the TB.

[0230] It should be noted that the above describes only some embodiments of the present invention, and modifications and / or variations thereof may be made without departing from the scope and spirit of the present invention, and the embodiments are illustrative and not restrictive.

Claims

1. 1. A method for decoding a coding unit from a bitstream, the coding unit being split from a coding tree unit of an image using a tree structure, the coding unit being capable of having at least a luma component or multiple chroma components, the multiple chroma components including a Cb component and a Cr component, the method comprising: a first decoding step of decoding, if the coding unit has the luma component, a luma transform skip flag for the luma component from the bitstream; a second decoding step of decoding, when the coding unit has the plurality of chroma components, a first chroma transform skip flag for the Cb component and a second chroma transform skip flag for the Cr component from the bitstream; a determining step for determining for the coding unit whether to decode an index for a particular transform process from the bitstream; a kernel to be used in the particular transformation process can be selected from a candidate set of multiple kernels, and the index is an index that identifies the kernel to be used; a third decoding step of decoding the index for the particular transform process from the bitstream for the coding unit according to the result of the determination in the determining step; and the luma transform skip flag indicates whether luma transform processing for the luma component is skipped; the first chroma transform skip flag indicates whether the first chroma transform process for the Cb component is skipped; the second chroma transform skip flag indicates whether the second chroma transform process for the Cr component is skipped; In a case where the coding tree unit has a size of 128×128, and a coding tree structure for the luma component in the coding tree unit is separate from coding tree structures for the multiple chroma components in the coding tree unit, (a) the coding tree unit is divided into four regions, each having a size of 64×64, common to the luma component and the multiple chroma components, (b) a dual tree structure for the luma component and a dual tree structure for the multiple chroma components start for each of the four regions, and (c) before the determination of whether to decode the index for each of the coding units divided from a certain region using the dual tree structure for the multiple chroma components is performed from the certain region, the determination of whether to decode the index for each of the coding units divided from the certain region using the dual tree structure for the luma component is performed; When the coding unit is divided from the coding tree unit using a single tree structure and each transform block in the coding unit has a significant coefficient only at a DC position, the specific transform process for the coding unit is never performed regardless of other conditions, and the index for the coding unit is never decoded from the bitstream; when the luma transform process, the first chroma transform process, and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the single tree structure, the index for the coding unit is not decoded from the bitstream and the value of the index for the coding unit is inferred to be 0; if the luma transform process is skipped and the coding unit is split from the coding tree unit using the dual tree structure for the luma component, the index for the coding unit is not decoded from the bitstream and a value of the index for the coding unit is inferred to be 0; When the first chroma transform process and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the dual tree structure for the multiple chroma components, the index for the coding unit is not decoded from the bitstream and the value of the index for the coding unit is inferred to be 0. A method characterized by:

2. The image has a 4:2:0 chroma format 2. The method of claim 1 .

3. When intra prediction and the single tree structure is used, the use of chroma blocks with sizes of 2x2, 2x4, or 4x2 is not permitted.

2. The method of claim 1 .

4. The index having a value of 0 indicates that the particular transformation is not used.

2. The method of claim 1 .

5. The DC position in a transform block is the upper left position among the multiple positions in the transform block.

2. The method of claim 1 .

6. The DC position in a transform block is the position that is scanned last in a predetermined scan order among multiple positions in the transform block.

2. The method of claim 1 .

7. The dual tree structure for the luma component is decoded for the certain region before the dual tree structure for the plurality of chroma components is decoded for the certain region.

2. The method of claim 1 .

8. If the transform block included in the coding unit has a significant coefficient, a flag indicating whether the magnitude of the significant coefficient is greater than 1 is decoded, and the magnitude of the significant coefficient is determined using the decoded flag.

2. The method of claim 1 .

9. Context-coded bins are used for the flags 9. The method of claim 8.

10. The sign information of the significant coefficient is decoded.

9. The method of claim 8.

11. The single tree structure is a tree structure in which the coding tree structure is common to the luma component and the plurality of chroma components.

2. The method of claim 1 .

12. 1. A method for encoding a coding unit into a bitstream, the coding unit being split from a coding tree unit of an image using a tree structure, the coding unit being capable of having at least a luma component or multiple chroma components, the multiple chroma components including a Cb component and a Cr component, the method comprising: a first encoding step of encoding a luma transform skip flag for the luma component into the bitstream if the coding unit has the luma component; a second encoding step of encoding, when the coding unit has the plurality of chroma components, a first chroma transform skip flag for the Cb component and a second chroma transform skip flag for the Cr component into the bitstream; a determining step for determining for the coding unit whether to code an index for a particular transformation process into the bitstream; a kernel to be used in the particular transformation process can be selected from a candidate set of multiple kernels, and the index is an index that identifies the kernel to be used; a third encoding step of encoding the index for the particular transform process into the bitstream for the encoding unit according to the result of the determination in the determining step; and the luma transform skip flag indicates whether luma transform processing for the luma component is skipped; the first chroma transform skip flag indicates whether the first chroma transform process for the Cb component is skipped; the second chroma transform skip flag indicates whether the second chroma transform process for the Cr component is skipped; In a case where the coding tree unit has a size of 128×128, and a coding tree structure for the luma component in the coding tree unit is separate from coding tree structures for the multiple chroma components in the coding tree unit, (a) the coding tree unit is divided into four regions, each having a size of 64×64, common to the luma component and the multiple chroma components, (b) a dual tree structure for the luma component and a dual tree structure for the multiple chroma components start for each of the four regions, and (c) before the determination is made as to whether to code the index for each of the coding units divided from a certain region using the dual tree structure for the multiple chroma components from the certain region, the determination is made as to whether to code the index for each of the coding units divided from the certain region using the dual tree structure for the luma component; When the coding unit is divided from the coding tree unit using a single tree structure, and each transform block in the coding unit has a significant coefficient only at a DC position, the specific transform process for the coding unit is never performed regardless of other conditions, and the index for the coding unit is never coded into the bitstream; when the luma transform process, the first chroma transform process, and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the single tree structure, the index for the coding unit is not coded into the bitstream, and the value of the index for the coding unit is estimated to be 0; when the luma transform process is skipped and the coding unit is split from the coding tree unit using the dual tree structure for the luma component, the index for the coding unit is not coded into the bitstream and the value of the index for the coding unit is inferred to be 0; When the first chroma transform process and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the dual tree structure for the multiple chroma components, the index for the coding unit is not coded into the bitstream and the value of the index for the coding unit is estimated to be 0. A method characterized by:

13. The image has a 4:2:0 chroma format 13. The method of claim 12.

14. When intra prediction and the single tree structure is used, the use of chroma blocks with sizes of 2x2, 2x4, or 4x2 is not permitted.

13. The method of claim 12.

15. The index having a value of 0 indicates that the particular transformation is not used.

13. The method of claim 12.

16. The DC position in a transform block is the upper left position among the multiple positions in the transform block.

13. The method of claim 12.

17. The DC position in a transform block is the position that is scanned last in a predetermined scan order among multiple positions in the transform block.

13. The method of claim 12.

18. The dual tree structure for the luma component is encoded for the certain region before the dual tree structure for the plurality of chroma components is encoded for the certain region.

13. The method of claim 12.

19. If the transform block included in the coding unit has a significant coefficient, a flag indicating whether the magnitude of the significant coefficient is greater than 1 is coded using a context bin.

13. The method of claim 12.

20. The sign information of the significant coefficient is encoded.

20. The method of claim 19.

21. The single tree structure is a tree structure in which the coding tree structure is common to the luma component and the plurality of chroma components.

13. The method of claim 12.

22. 1. An apparatus for decoding a coding unit from a bitstream, the coding unit being split from a coding tree unit of an image using a tree structure, the coding unit being capable of having at least a luma component or multiple chroma components, the multiple chroma components including a Cb component and a Cr component, the apparatus comprising: a first decoding means for decoding, when the coding unit has the luma component, a luma transform skip flag for the luma component from the bitstream; second decoding means for decoding, when the coding unit has the plurality of chroma components, a first chroma transform skip flag for the Cb component and a second chroma transform skip flag for the Cr component from the bitstream; a determining means for determining whether the encoding unit should decode an index for a specific transformation process from the bitstream; a kernel to be used in the particular transformation process can be selected from a candidate set of multiple kernels, and the index is an index that identifies the kernel to be used; a third decoding means for decoding the index for the specific transform process for the coding unit from the bitstream according to a result of the determination by the determining means; and the luma transform skip flag indicates whether luma transform processing for the luma component is skipped; the first chroma transform skip flag indicates whether the first chroma transform process for the Cb component is skipped; the second chroma transform skip flag indicates whether the second chroma transform process for the Cr component is skipped; In a case where the coding tree unit has a size of 128×128, and a coding tree structure for the luma component in the coding tree unit is separate from coding tree structures for the multiple chroma components in the coding tree unit, (a) the coding tree unit is divided into four regions, each having a size of 64×64, common to the luma component and the multiple chroma components, (b) a dual tree structure for the luma component and a dual tree structure for the multiple chroma components start for each of the four regions, and (c) before the determination of whether to decode the index for each of the coding units divided from a certain region using the dual tree structure for the multiple chroma components is performed from the certain region, the determination of whether to decode the index for each of the coding units divided from the certain region using the dual tree structure for the luma component is performed; When the coding unit is divided from the coding tree unit using a single tree structure and each transform block in the coding unit has a significant coefficient only at a DC position, the specific transform process for the coding unit is never performed regardless of other conditions, and the index for the coding unit is never decoded from the bitstream; when the luma transform process, the first chroma transform process, and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the single tree structure, the index for the coding unit is not decoded from the bitstream and the value of the index for the coding unit is inferred to be 0; if the luma transform process is skipped and the coding unit is split from the coding tree unit using the dual tree structure for the luma component, the index for the coding unit is not decoded from the bitstream and a value of the index for the coding unit is inferred to be 0; When the first chroma transform process and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the dual tree structure for the multiple chroma components, the index for the coding unit is not decoded from the bitstream and the value of the index for the coding unit is inferred to be 0. An apparatus characterized in that

23. 1. An apparatus for encoding a coding unit into a bitstream, the coding unit being divided from a coding tree unit of an image using a tree structure, the coding unit being capable of having at least a luma component or multiple chroma components, the multiple chroma components including a Cb component and a Cr component, the apparatus comprising: a first encoding means for encoding a luma transform skip flag for the luma component into the bitstream when the coding unit has the luma component; a second encoding means for encoding, when the encoding unit has the plurality of chroma components, a first chroma transform skip flag for the Cb component and a second chroma transform skip flag for the Cr component into the bitstream; a determining means for determining whether to encode an index for a particular transformation process into the bitstream for the encoding unit; a kernel to be used in the particular transformation process can be selected from a candidate set of multiple kernels, and the index is an index that identifies the kernel to be used; a third encoding means for encoding the index for the specific transformation process into the bitstream for the coding unit according to the result of the determination by the determining means; and the luma transform skip flag indicates whether luma transform processing for the luma component is skipped; the first chroma transform skip flag indicates whether the first chroma transform process for the Cb component is skipped; the second chroma transform skip flag indicates whether the second chroma transform process for the Cr component is skipped; In a case where the coding tree unit has a size of 128×128, and a coding tree structure for the luma component in the coding tree unit is separate from coding tree structures for the multiple chroma components in the coding tree unit, (a) the coding tree unit is divided into four regions, each having a size of 64×64, common to the luma component and the multiple chroma components, (b) a dual tree structure for the luma component and a dual tree structure for the multiple chroma components start for each of the four regions, and (c) before the determination is made as to whether to code the index for each of the coding units divided from a certain region using the dual tree structure for the multiple chroma components from the certain region, the determination is made as to whether to code the index for each of the coding units divided from the certain region using the dual tree structure for the luma component; When the coding unit is divided from the coding tree unit using a single tree structure, and each transform block in the coding unit has a significant coefficient only at a DC position, the specific transform process for the coding unit is never performed regardless of other conditions, and the index for the coding unit is never coded into the bitstream; when the luma transform process, the first chroma transform process, and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the single tree structure, the index for the coding unit is not coded into the bitstream, and the value of the index for the coding unit is estimated to be 0; when the luma transform process is skipped and the coding unit is split from the coding tree unit using the dual tree structure for the luma component, the index for the coding unit is not coded into the bitstream and the value of the index for the coding unit is inferred to be 0; When the first chroma transform process and the second chroma transform process are skipped and the coding unit is split from the coding tree unit using the dual tree structure for the multiple chroma components, the index for the coding unit is not coded into the bitstream and the value of the index for the coding unit is estimated to be 0. An apparatus characterized in that

24. A program causing a computer to execute the method according to any one of claims 1 to 11.

25. A program causing a computer to execute the method according to any one of claims 12 to 21.

Images