Adaptive scaling list control for video decoding

Abstract

One example method includes: decoding an explicitly defined scaling list from a decoded video bitstream; determining a set of block types to which the explicitly defined scaling list is eligible to be applied, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and applying the explicitly defined scaling list to blocks included in the block type set.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Application No. 17 / 225,643, filed April 8, 2021, and U.S. Provisional Application No. 63 / 012,794, filed April 20, 2020, the entire contents of which are incorporated herein by reference. U.S. Application No. 17 / 225,643 claims the benefit of U.S. Provisional Application No. 63 / 012,794, filed April 20, 2020. Technical Field

[0003] This disclosure relates to video encoding and video decoding. Background Technology

[0004] Digital video capabilities can be integrated into a variety of devices, including digital televisions, digital live broadcast systems, wireless broadcasting systems, personal digital assistants (PDAs), laptops or desktop computers, tablets, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radio phones, so-called "smartphones," video conferencing equipment, and video streaming devices. Digital video devices implement video decoding technologies, such as those described in standards defined by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10 Advanced Video Decoding (AVC), ITU-T H.265 / High-Efficiency Video Decoding (HEVC), and extensions to these standards. By implementing such video decoding technologies, video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0005] Video decoding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or remove inherent redundancy in video sequences. For block-based video decoding, video slices (e.g., video pictures or portions of video pictures) can be segmented into video blocks, which may also be referred to as decoding tree units (CTUs), decoding units (CUs), and / or decoding nodes. Video blocks in an intra-frame decoding (I) slice of a picture are encoded using spatial prediction relative to reference samples in adjacent blocks within the same picture. Video blocks in an inter-frame decoding (P or B) slice of a picture can use spatial prediction relative to reference samples in adjacent blocks within the same picture or temporal prediction relative to reference samples in other reference pictures. A picture may be referred to as a frame, and a reference picture may be referred to as a reference frame. Summary of the Invention

[0006] This disclosure generally describes techniques for adaptive scaling list control. Video decoders (e.g., video encoders and / or video decoders) can apply scaling lists for dequantization. For example, a video decoder can use a scaling list to scale the transform coefficients of a video data block. In some examples, a video decoder can apply adaptive color transformation (ACT). For example, a video decoder can transform the color space of the residual signal from a first color space (e.g., RGB) to a second color space (e.g., YCgCo). Applying a combination of scaling lists and ACT can have one or more drawbacks. For example, in general, the correlation between the three components of the residual signal in the RGB domain is stronger than that in the YCgCo domain, meaning that the energy distribution of the transform coefficients in the RGB and YCgCo domains will be different. Some codecs require that the scaling list is always shared for G and Y, B and Cg, R and Co. However, such sharing of the scaling list can lead to undesirable behavior in the quantization process. One solution to overcome such undesirable behavior is to disable the application of the scaling list to blocks to which ACT has been applied (i.e., ACT blocks). However, this solution may have its own drawbacks. For example, it might still be desirable to allow some scaling lists to be applied to ACT blocks.

[0007] According to one or more techniques of this disclosure, a video decoder can explicitly signal the set of block types to which a scaling list is eligible. For example, if the video decoder determines that the scaling list is not eligible for application to the ACT block type (e.g., the block type set does not include the ACT block type), the video decoder may not (e.g., may suppress) scaling the transform coefficients of the video data blocks to which ACT is applied. If the video decoder determines that the scaling list is eligible for application to the ACT block type, the video decoder can use the scaling list to scale the transform coefficients of the video data blocks to which ACT is applied. Therefore, control over the application of the scaling list can be adaptive. In this way, the video decoder can avoid the disadvantages of scaling list sharing while retaining the ability to apply the scaling list to ACT blocks.

[0008] In one example, one approach includes: decoding an explicitly defined scaling list from a decoded video bitstream; determining a set of block types to which the explicitly defined scaling list is eligible to be applied, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and applying the explicitly defined scaling list to blocks included in the block type set.

[0009] In another example, an apparatus includes: a memory configured to store at least a portion of a video bitstream; and processing circuitry configured to: decode an explicitly defined scaling list from the decoded video bitstream; determine a set of block types to which the explicitly defined scaling list is eligible to be applied based on the values ​​of one or more syntax elements decoded from the SPS of the decoded video bitstream; and apply the explicitly defined scaling list to blocks included in the block type set.

[0010] In another example, one approach includes: encoding an explicitly defined scaling list in a decoded video bitstream; encoding one or more syntax elements of a block type set to which the explicitly defined scaling list is eligible in an SPS of the decoded video bitstream; and applying the explicitly defined scaling list to blocks included in the block type set.

[0011] In another example, an apparatus includes: a memory configured to store at least a portion of a video bitstream; and processing circuitry configured to encode an explicitly defined scaling list in the decoded video bitstream; encode one or more syntax elements in an SPS of the decoded video bitstream to which the explicitly defined scaling list is eligible; and apply the explicitly defined scaling list to blocks included in the block type set.

[0012] In another example, an apparatus includes: components for decoding an explicitly defined scaling list from a decoded video bitstream; components for determining a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from the decoded video bitstream using SPS; and components for applying the explicitly defined scaling list to blocks included in the block type set.

[0013] In another example, an apparatus includes: components for encoding an explicitly defined scaling list in a decoded video bitstream; components for encoding one or more syntax elements of a block type set to which the explicitly defined scaling list is eligible in an SPS of the decoded video bitstream; and components for applying the explicitly defined scaling list to blocks included in the block type set.

[0014] In another example, a computer-readable storage medium stores instructions that, when executed, cause one or more processors to: decode an explicitly defined scaling list from a decoded video bitstream; determine a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from the SPS of the decoded video bitstream; and apply the explicitly defined scaling list to blocks included in the block type set.

[0015] In another example, a computer-readable storage medium stores instructions that, when executed, cause one or more processors to: encode an explicitly defined scaling list in a decoded video bitstream; encode one or more syntax elements in an SPS of the decoded video bitstream to which the explicitly defined scaling list is eligible; and apply the explicitly defined scaling list to blocks included in the block type set.

[0016] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0017] Figure 1 This is a block diagram illustrating an example video encoding and decoding system that can perform the techniques of this disclosure.

[0018] Figure 2A and Figure 2B This is a conceptual diagram showing an example quadtree binary tree (QTBT) structure and its corresponding decoding tree unit (CTU).

[0019] Figure 3 This is a block diagram illustrating an example video encoder that can perform the techniques of this disclosure.

[0020] Figure 4 This is a block diagram illustrating an example video decoder that can perform the techniques disclosed herein.

[0021] Figure 5 This is a flowchart illustrating an example method for encoding the current block.

[0022] Figure 6 This is a flowchart illustrating an example method for decoding the current block.

[0023] Figure 7 This is a flowchart illustrating an example method for adaptive scaling list control according to one or more techniques of this disclosure. Detailed Implementation

[0024] Figure 1 This is a block diagram illustrating an example video encoding and decoding system 100 capable of performing the techniques of this disclosure. The techniques of this disclosure generally relate to decoding (encoding and / or decoding) video data. In general, video data includes any data used for processing video. Therefore, video data can include raw, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata, such as signaling data.

[0025] like Figure 1As shown, in this example, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. Specifically, source device 102 provides video data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can include any of a variety of devices, including desktop computers, laptop computers, mobile devices, tablet computers, set-top boxes, handsets such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, broadcast receiver devices, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication and may therefore be referred to as wireless communication devices.

[0026] exist Figure 1 In the example, source device 102 includes a video source 104, memory 106, video encoder 200, and output interface 108. Destination device 116 includes an input interface 122, video decoder 300, memory 120, and display device 118. According to this disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 can be configured to apply techniques for adaptive scaling list control. Source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, the source and destination devices may include other components or arrangements. For example, source device 102 may receive video data from an external video source such as an external camera. Similarly, destination device 116 may interface with an external display device instead of including an integrated display device.

[0027] like Figure 1 The system 100 shown is merely an example. Generally, any digital video encoding and / or decoding device can perform techniques for adaptive scaling list control. Source device 102 and destination device 116 are merely examples of such decoding devices, where source device 102 generates decoded video data for transmission to destination device 116. In this disclosure, "decoding device" refers to a device that performs the decoding (encoding and / or decoding) of data. Thus, video encoder 200 and video decoder 300 represent examples of decoding devices, specifically, a video encoder and a video decoder, respectively. In some examples, source device 102 and destination device 116 may operate in a substantially symmetrical manner, such that each of source device 102 and destination device 116 includes video encoding and decoding components. Therefore, system 100 can support one-way or two-way video transmission between source device 102 and destination device 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0028] Generally, video source 104 represents the source of video data (i.e., raw, unencoded video data) and provides a series of consecutive pictures (also called “frames”) of video data to video encoder 200, which encodes the data of the pictures. Video source 104 of source device 102 may include video capture devices such as cameras, video archives containing previously captured raw video, and / or video feed interfaces that receive video from video content providers. As a further alternative, video source 104 may generate computer graphics-based data as source video, or a combination of live video, archived video, and computer-generated video. In each case, video encoder 200 encodes the captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange the pictures from the order of receipt (sometimes referred to as “display order”) into a decoding order for decoding. Video encoder 200 may generate a bitstream comprising the encoded video data. Then, the source device 102 can output the encoded video data to the computer-readable medium 110 via the output interface 108 for reception and / or acquisition by, for example, the input interface 122 of the destination device 116.

[0029] The memory 106 of source device 102 and the memory 120 of destination device 116 represent general-purpose memory. In some examples, memories 106 and 120 may store raw video data, such as raw video from video source 104 and raw decoded video data from video decoder 300. Additionally or alternatively, memories 106 and 120 may store software instructions executable by, for example, video encoder 200 and video decoder 300, respectively. Although memories 106 and 120 are shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 may also include functionally similar or equivalent internal memories. Furthermore, memories 106 and 120 may store, for example, encoded video data output from video encoder 200 and input to video decoder 300. In some examples, portions of memories 106 and 120 may be allocated as one or more video buffers, for example, to store raw, decoded, and / or encoded video data.

[0030] Computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded video data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium that enables source device 102 to directly transmit encoded video data to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, output interface 108 can demodulate the transmitted signal including the encoded video data, and input interface 122 can demodulate the received transmitted signal. The communication medium can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network (such as a local area network, a wide area network, or a global network such as the Internet). The communication medium can include a router, switch, base station, or any other equipment that facilitates communication from source device 102 to destination device 116.

[0031] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 can include any of a variety of distributed or locally accessed data storage media, such as hard disks, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0032] In some examples, source device 102 may output encoded video data to file server 114 or another intermediate storage device that may store the encoded video data generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing encoded video data and sending such encoded video data to destination device 116. File server 114 may represent (e.g., for a website) a web server, file transfer protocol (FTP) server, content delivery network device, or network attached storage (NAS) device. Destination device 116 may access the encoded video data from file server 114 via any standard data connection including an Internet connection. This may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for accessing encoded video data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a download transfer protocol, or a combination thereof.

[0033] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data such as encoded video data according to cellular communication standards such as 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc. In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to operate according to specifications such as IEEE 802.11, IEEE 802.15 (e.g., ZigBee). TM ),Bluetooth TM Other wireless standards, such as standard or similar standards, are used to transmit data such as encoded video data. In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include an SoC device performing functions belonging to video encoder 200 and / or output interface 108, and destination device 116 may include an SoC device performing functions belonging to video decoder 300 and / or input interface 122.

[0034] The technology disclosed herein can be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (such as Dynamic Adaptive Streaming (DASH) via HTTP), digital video encoded onto a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0035] The input interface 122 of the destination device 116 receives an encoded video bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200, such as syntax elements having values ​​describing the characteristics and / or processing of video blocks or other decoding units (e.g., slices, pictures, picture groups, sequences, etc.), which is also used by the video decoder 300. The display device 118 displays decoded images of the decoded video data to the user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0036] Although not in Figure 1As shown, but in some examples, the video encoder 200 and video decoder 300 may each be integrated with the audio encoder and / or audio decoder, and may include appropriate MUX-DEMUX units or other hardware and / or software to process multiplexed streams that include both audio and video in a common data stream. Where applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol or other protocols (such as User Datagram Protocol (UDP)).

[0037] The video encoder 200 and video decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for software in a suitable non-transitory computer-readable medium and use one or more processors to execute those instructions in hardware to perform the technology of this disclosure. Each of the video encoder 200 and video decoder 300 may be included in one or more encoders or decoders, any one of which may be integrated as part of a combined encoder / decoder (codec) in the respective device. Devices including the video encoder 200 and / or video decoder 300 may include integrated circuits, microprocessors, and / or wireless communication devices such as cellular phones.

[0038] The video encoder 200 and video decoder 300 can operate according to video decoding standards such as ITU-T H.265 (also known as High Efficiency Video Decoding (HEVC)) or its extensions (such as Multi-View and / or Scalable Video Decoding Extensions). Alternatively, the video encoder 200 and video decoder 300 can operate according to other proprietary or industry standards such as ITU-T H.266 (also known as Versatile Video Decoding (VVC)). The latest draft of the VVC standard is described in "Versatile Video Coding (Draft 8)" by Bross et al. (ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 17th meeting of the Joint Video Experts Group (JVET): Brussels, Belgium, 7-17 January 2020, JVET-Q2001-vE) (hereinafter referred to as "VVC Draft 8"). However, the technology disclosed herein is not limited to any particular decoding standard.

[0039] Generally, video encoder 200 and video decoder 300 can perform block-based decoding of images. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used in the encoding and / or decoding process). For example, a block may include a two-dimensional matrix of samples of luminance and / or chrominance data. Generally, video encoder 200 and video decoder 300 can decode video data represented in YUV (e.g., Y, Cb, Cr) format. That is, video encoder 200 and video decoder 300 can decode luminance and chrominance components, rather than decoding red, green, and blue (RGB) data of samples of an image, where chrominance components may include both red and blue chrominance components. In some examples, video encoder 200 converts the received RGB formatted data to a YUV representation before encoding, and video decoder 300 converts the YUV representation to RGB format. Alternatively, preprocessing and post-processing units (not shown) can perform these conversions.

[0040] This disclosure can generally relate to the decoding (e.g., encoding and decoding) of images, including the process of encoding or decoding image data. Similarly, this disclosure can relate to the decoding of blocks of images, including the process of encoding or decoding block data, such as prediction and / or residual decoding. Encoded video bitstreams typically include a series of values ​​for syntax elements representing decoding decisions (e.g., decoding modes) and image-to-block segmentation. Therefore, references to decoding images or blocks should generally be understood as decoding the values ​​of the syntax elements used to form images or blocks.

[0041] HEVC defines various blocks, including decoding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder (such as a video encoder 200) partitions a decoding tree unit (CTU) into CUs based on a quadtree structure. That is, the video decoder partitions the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. Nodes without child nodes can be called "leaf nodes," and the CU of such leaf nodes can include one or more PUs and / or one or more TUs. The video decoder can also partition PUs and TUs. For example, in HEVC, a residual quadtree (RQT) represents a partition of a TU. In HEVC, a PU represents inter-frame prediction data, while a TU represents residual data. CUs with intra-frame prediction include intra-frame prediction information, such as intra-frame mode indication.

[0042] As another example, video encoder 200 and video decoder 300 can be configured to operate according to VVC. According to VVC, the video decoder (such as video encoder 200) segments the image into multiple decoding tree units (CTUs). Video encoder 200 can segment the CTUs according to a tree structure, such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure removes the concept of multiple segmentation types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level of segmentation based on quadtree segmentation and a second level of segmentation based on binary tree segmentation. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to decoding units (CUs).

[0043] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) partitioning (also known as tripartite tree (TT)) partitioning. A ternary or tripartite tree partition is a partition that divides a block into three sub-blocks. In some examples, a ternary or tripartite tree partition divides a block into three sub-blocks without partitioning the original block through a center. The partitioning types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0044] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma and chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for the respective chroma components).

[0045] The video encoder 200 and video decoder 300 can be configured to use quadtree segmentation, QTBT segmentation, MTT segmentation, or other segmentation structures according to HEVC. For illustrative purposes, the description of the techniques of this disclosure is based on QTBT segmentation. However, it should be understood that the techniques of this disclosure can also be applied to video decoders configured to use quadtree segmentation or other types of segmentation.

[0046] In some examples, a CTU includes a decoded tree block (CTB) of luminance samples, two corresponding CTBs of chrominance samples of an image with three sample arrays, or a CTB of samples of a monochrome image or an image decoded using three separate color planes and a syntax structure for decoding the samples. A CTB can be an N×N block of samples for some value N, such that dividing a component into a CTB is a partition. A component is an array or a single sample from one of the three arrays (luminance and two chrominance) constituting an image in a 4:2:0, 4:2:2, or 4:4:4 color format, or an array or a single sample constituting an array or array constituting an image in monochrome format. In some examples, a decoded block is an M×N block of samples for some values ​​of M and N, such that dividing a CTB into a decoded block is a partition.

[0047] These blocks (e.g., CTUs or CUs) can be grouped in various ways within an image. As an example, a brick can refer to a rectangular area of ​​a row of CTUs within a specific tile in an image. A tile can be a rectangular area of ​​CTUs within a specific tile column and a specific tile row in an image. A tile column is a rectangular area of ​​CTUs with a height equal to the height of the image and a width specified by a syntax element (e.g., such as in the image parameter set). A tile row is a rectangular area of ​​CTUs with a height specified by a syntax element (e.g., such as in the image parameter set) and a width equal to the width of the image.

[0048] In some examples, a tile can be divided into multiple bricks, each of which may include one or more CTU rows within the tile. A tile that is not divided into multiple bricks can also be referred to as a brick. However, bricks that are a true subset of a tile cannot be referred to as a tile.

[0049] Bricks in an image can also be arranged in slices. A slice can be an integer number of bricks in an image that can be exclusively contained within a single Network Abstraction Layer (NAL) unit. In some examples, a slice consists of a continuous sequence of multiple complete tiles or a single complete tile.

[0050] This disclosure uses "N×N" and "N multiplied by N" interchangeably to refer to the sample dimensions of a block (such as a CU or other video block) in terms of both vertical and horizontal dimensions, for example, 16×16 samples or 16 by 16 samples. Generally, a 16×16 CU will have 16 samples in the vertical direction (y = 16) and 16 samples in the horizontal direction (x = 16). Similarly, an N×N CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. Samples in a CU can be arranged in rows and columns. Furthermore, a CU does not necessarily need to have the same number of samples in the horizontal direction as it does in the vertical direction. For example, a CU can include N×M samples, where M is not necessarily equal to N.

[0051] The video encoder 200 encodes video data for representation prediction and / or residual information for the CU, as well as other information. The prediction information indicates how the CU will be predicted to form a prediction block for the CU. The residual information typically represents the sample-by-sample difference between the samples of the CU before encoding and the prediction block.

[0052] To predict the Cubic Frame (CU), the video encoder 200 typically forms prediction blocks for the CU using either inter-frame prediction or intra-frame prediction. Inter-frame prediction generally refers to predicting the CU from data of a previously decoded image, while intra-frame prediction generally refers to predicting the CU from data of a previously decoded image of the same frame. To perform inter-frame prediction, the video encoder 200 can use one or more motion vectors to generate prediction blocks. The video encoder 200 can typically perform motion search to identify a reference block that closely matches the CU, for example, based on the difference between the CU and a reference block. The video encoder 200 can use sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), or other such difference calculations to compute difference metrics to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 can use unidirectional or bidirectional prediction to predict the current CU.

[0053] Some examples of VVC also provide an affine motion compensation mode, which can be considered an inter-frame prediction mode. In affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion such as zooming in or out, rotation, perspective motion, or other irregular motion types.

[0054] To perform intra-frame prediction, the video encoder 200 can select an intra-frame prediction mode to generate prediction blocks. Some examples of VVC provide sixty-seven intra-frame prediction modes, including various orientation modes, as well as planar and DC modes. Generally, the video encoder 200 selects an intra-frame prediction mode that describes the neighboring samples of the current block (e.g., the block of the CU) to predict samples of the current block. Assuming the video encoder 200 decodes the CTU and CU in raster scan order (from left to right, from top to bottom), such samples can typically be located above, to the upper left, or to the left of the current block in the same image as the current block.

[0055] The video encoder 200 encodes data representing the prediction mode used for the current block. For example, for inter-frame prediction modes, the video encoder 200 may encode data representing which of the various available inter-frame prediction modes is used, and the motion information used for the corresponding mode. For example, for unidirectional or bidirectional inter-frame prediction, the video encoder 200 may encode motion vectors using Advanced Motion Vector Prediction (AMVP) or merge modes. For affine motion compensation modes, the video encoder 200 may use similar modes to encode motion vectors.

[0056] Following the prediction of a block, such as after intra-frame or inter-frame prediction of the block, the video encoder 200 can compute residual data for the block. Residual data, such as a residual block, represents the sample-by-sample difference between the block and the predicted block formed using the corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to produce transformed data in the transform domain rather than the sample domain. For example, the video encoder 200 can apply a Discrete Cosine Transform (DCT), integer transform, wavelet transform, or conceptually similar transforms to the residual video data. Additionally, the video encoder 200 can apply secondary transforms after the first transform, such as Mode Correlated Inseparable Secondary Transform (MDNSST), Signal Correlation Transform, Karhunen-Loeve Transform (KLT), etc. The video encoder 200 produces transform coefficients after applying one or more transforms.

[0057] As described above, after any transformation used to generate the transform coefficients, the video encoder 200 can perform quantization on the transform coefficients. Quantization generally refers to the process of quantizing the transform coefficients to reduce the amount of data used to represent them, thereby providing further compression. By performing the quantization process, the video encoder 200 can reduce the bit depth associated with some or all of the transform coefficients. For example, the video encoder 200 can round an n-bit value down to an m-bit value during quantization, where n is greater than m. In some examples, to perform quantization, the video encoder 200 can perform a bit-by-bit right shift of the value to be quantized.

[0058] After quantization, the video encoder 200 can scan the transform coefficients to generate a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan can be designed to place higher-energy (and therefore lower-frequency) transform coefficients before the vector and lower-energy (and therefore higher-frequency) transform coefficients after the vector. In some examples, the video encoder 200 can utilize a predefined scan order to scan the quantized transform coefficients to generate a serialized vector, and then entropy-encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 can perform an adaptive scan. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 can entropy-encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic decoding (CABAC). The video encoder 200 can also entropy-encode the values ​​of syntax elements describing metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0059] To perform CABAC, the video encoder 200 can assign context within a context model to the symbols to be transmitted. The context may involve, for example, whether the neighboring values ​​of a symbol are zero. Probability determination can be based on the context assigned to the symbols.

[0060] The video encoder 200 can further generate, for example, block-based syntax data, picture-based syntax data, and sequence-based syntax data, or other syntax data such as sequence parameter sets (SPS), picture parameter sets (PPS), or video parameter sets (VPS) for the video decoder 300 in image headers, block headers, and slice headers. The video decoder 300 can also decode such syntax data to determine how to decode the corresponding video data.

[0061] In this way, the video encoder 200 can generate a bitstream that includes encoded video data, such as syntax elements describing the segmentation of images into blocks (e.g., CUs) and prediction and / or residual information for the blocks. Finally, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0062] Generally, the video decoder 300 performs the reverse process of the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 can use CABAC, although in a substantially similar manner to the CABAC encoding process of the video encoder 200, to decode the values ​​of the bitstream for syntax elements. Syntax elements can define segmentation information for image-to-CTU segmentation, and segmentation of each CTU according to a corresponding segmentation structure such as a QTBT structure, to define the CU of the CTU. Syntax elements can further define prediction and residual information for video data blocks (e.g., CUs).

[0063] The residual information can be represented, for example, by quantized transform coefficients. The video decoder 300 can inversely quantize and inversely transform the quantized transform coefficients of the block to reconstruct the residual block for that block. The video decoder 300 uses a signaling prediction mode (intra-frame or inter-frame prediction) and associated prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for that block. The video decoder 300 can then combine the prediction block and the residual block (on a sample-by-sample basis) to reconstruct the original block. The video decoder 300 can perform additional processing, such as performing a deblocking process to reduce visual artifacts along the block boundaries.

[0064] In VVC draft 8, user-defined scaling lists can be applied for dequantization.

[0065] Also in VVC Draft 8, Adaptive Color Transform (ACT) can be applied to 4:4:4 RGB video signals. ACT can be controlled at the CU level flag (cu_act_enabled_flag). When a video decoder (e.g., video encoder 200 and / or video decoder 300) applies ACT to the current CU, the video decoder can convert the color space of the residual signal from RGB (GBR) to YCgCo. As suggested by Iwamura et al., “Scaling list for adaptive color transform” (Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 18th meeting: via teleconference, April 15-24, 2020, JVET-R0380v3 (hereinafter “JVET-R0380”), generally speaking, the correlation between the residual signals of the three components in the RGB domain is stronger than that in the YCgCo domain, which means that the energy distribution of the transform coefficients in the RGB and YCgCo domains will be different. In VVC Draft 8, the scaling list is always shared for G and Y, for B and Cg, and for R and Co, which may lead to undesirable behavior in the quantization process. JVET-R0380 recommends disabling the scaling matrix for blocks decoded with ACT. However, there may be situations where scaling lists are expected to be used for blocks that are ACT-decoded but not for blocks that are not ACT-decoded.

[0066] This disclosure describes a technique for adaptive scaling list control for Adaptive Color Transformation (ACT). The video decoder can receive a user-specified variable, ScalingListType, which indicates which types of decoded blocks the scaling list can be applied to. As an example, a ScalingListType equal to a first value (e.g., SCALING_LIST_TYPE_ACT) can indicate that the scaling list can be applied only to blocks decoded by ACT, and not to other blocks. As another example, a ScalingListType equal to a second value (e.g., SCALING_LIST_TYPE_OTHER) can indicate that the scaling list is not applied to blocks decoded by ACT, but can be applied to other blocks. Other signaling techniques can be used for the same purpose (e.g., to indicate which types of decoded blocks the scaling list can be applied to).

[0067] The techniques described above can extend the scaling list used to control blocks decoded by Low Frequency Inseparable Transform (LFNST). As an example, a variable `ScalingListType` equal to a first value (e.g., `SCALING_LIST_TYPE_ACT`) can indicate that the scaling list can be applied only to blocks decoded by ACT, and not to other blocks. As another example, a variable `ScalingListType` equal to a second value (e.g., `SCALING_LIST_TYPE_LFNST`) can indicate that the scaling list can be applied only to blocks decoded by LFNST, and not to other blocks. As yet another example, a variable `ScalingListType` equal to a third value (e.g., `SCALING_LIST_TYPE_OTHER`) can indicate that the scaling list is not applied to blocks decoded by ACT, nor to blocks decoded by LFNST, but can be applied to other blocks.

[0068] The video decoder can signal the value of the variable `ScalingListType` in any parameter set or header structure. For example, the syntax element representing the value of `ScalingListType` can be signaled in a Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), Slice Header (SH), Adaptive Parameter Set (APS), or any other such structure. The video decoder can apply some constraints to the signaling of `ScalingListType`. As an example, if ACT is not enabled (e.g., if `cu_act_enabled_flag` indicates that ACT is not enabled), the video decoder can infer that the value of `ScalingListType` is not equal to `SCALING_LIST_TYPE_ACT`. As another example, if LFNST is not enabled, the video decoder can infer that the value of `ScalingListType` is not equal to `SCALING_LIST_TYPE_LFNST`.

[0069] Note that whether to apply the scaling list can also be controlled by other advanced syntax elements, such as the on / off control at the Sequence Parameter Set (SPS) and / or the Picture Header (PH).

[0070] According to the technology disclosed herein, a video decoder can obtain a scaling list (e.g., encode or decode one or more syntax elements representing an explicit scaling list); and decode syntax elements of a set of block types to which a specified scaling list can be applied via the decoded video bitstream.

[0071] This disclosure can generally refer to "signaling" certain information, such as syntax elements. The term "signaling" can generally refer to communication of values ​​for syntax elements and / or other data used for decoding encoded video data. That is, video encoder 200 can signal values ​​for syntax elements within the bitstream. Generally, signaling involves generating values ​​within the bitstream. As described above, source device 102 can transmit the bitstream to destination device 116 substantially in real-time or non-real-time, such as when syntax elements are stored in storage device 112 for subsequent retrieval by destination device 116.

[0072] Figure 2A and Figure 2B This is a conceptual diagram illustrating an example Quadtree Binary Tree (QTBT) structure 130 and its corresponding Decoding Tree Unit (CTU) 132. Solid lines represent quadtree partitions, and dashed lines indicate binary tree partitions. In each partition node (i.e., a non-leaf node) of the binary tree, a flag is signaled to indicate which partition type (i.e., horizontal or vertical) is used, where in this example, 0 indicates a horizontal partition and 1 indicates a vertical partition. For quadtree partitions, since quadtree nodes divide blocks horizontally and vertically into four sub-blocks of equal size, there is no need to indicate the partition type. Accordingly, the video encoder 200 can encode syntax elements (such as partition information) for the region tree level (i.e., solid lines) of the QTBT structure 130 and syntax elements (such as partition information) for the prediction tree level (i.e., dashed lines) of the QTBT structure 130, and the video decoder 300 can decode these syntax elements. The video encoder 200 can encode video data (such as prediction and transform data) for a CU represented by the terminal leaf nodes of the QTBT structure 130, and the video decoder 300 can decode the video data.

[0073] Generally speaking, Figure 2B The CTU 132 can be associated with parameters that define the size of the block corresponding to the nodes of the QTBT structure 130 at the first and second levels. These parameters can include the CTU size (representing the size of the CTU 132 in the sample), the minimum quadtree size (MinQTSize, representing the minimum allowed size of the leaf nodes of the quadtree), the maximum binary tree size (MaxBTSize, representing the maximum allowed size of the root node of the binary tree), the maximum binary tree depth (MaxBTDepth, representing the maximum allowed depth of the binary tree), and the minimum binary tree size (MinBTSize, representing the minimum allowed size of the leaf nodes of the binary tree).

[0074] The root node of the QTBT structure corresponding to CTU can have four child nodes at the first level of the QTBT structure, each of which can be partitioned according to a quadtree partition. That is, nodes at the first level are either leaf nodes (with no child nodes) or have four child nodes. An example of QTBT structure 130 represents such nodes as including a parent node and child nodes with solid lines for branching. If the nodes at the first level are not larger than the maximum allowed binary tree root node size (MaxBTSize), then these nodes can be further partitioned by the corresponding binary tree. A binary tree partition of a node can be performed iteratively until the partitioned nodes reach the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). An example of QTBT structure 130 represents such nodes with dashed lines for branching. The binary tree leaf nodes are called decoding units (CUs), which are used for prediction (e.g., intra-image prediction or inter-image prediction) and transformation without any further partitioning. As discussed above, CUs can also be referred to as “video chunks” or “blocks”.

[0075] In one example of a QTBT segmentation structure, the CTU size is set to 128×128 (luminance sample and two corresponding 64×64 chrominance samples), MinQTSize is set to 16×16, MaxBTSize is set to 64×64, MinBTSize (for both width and height) is set to 4, and MaxBTDepth is set to 4. First, a quadtree segmentation is applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can have sizes ranging from 16×16 (i.e., MinQTSize) to 128×128 (i.e., the CTU size). If a quadtree leaf node is 128×128, it will not be further segmented by a binary tree because this size exceeds MaxBTSize (i.e., 64×64 in this example). Otherwise, the quadtree leaf node will be further segmented by the binary tree. Therefore, the quadtree leaf node is also the root node of the binary tree and has a binary tree depth of 0. When the depth of the binary tree reaches MaxBTDepth (4 in this example), further partitioning is not allowed. Similarly, when the width of a binary tree node equals MinBTSize (4 in this example), further vertical partitioning is not allowed. Likewise, when the height of a binary tree node equals MinBTSize, further horizontal partitioning of that node is not allowed. As described above, the leaf nodes of the binary tree are called CUs and are further processed according to predictions and transformations without further partitioning.

[0076] Video encoder 200 and video decoder 300 can be configured to process quantization parameters. The QP value is used to determine the step size used for quantizing / inverse quantizing the coefficients. The QP value is specified in the range of -QpBdOffset to 63 (inclusive), where 63 is the maximum QP value. QpBdOffset is specified as a fixed value for a specific bit depth, derived as 6 * (bit depth – 8). The QP prime value is calculated by adding QpBdOffset to the specified QP value; this prime value is used to derive the actual step size. For ease of description, QP and QP prime values ​​can be used interchangeably in the remainder of the description. It should be understood that the QP value can be used only in most of the QP derivation process, while the QP prime value can be used only in the final stage before determining the step size. A change of 1 in the QP value roughly indicates a 12% change in step size, and a change of 6 in the QP value corresponds to a 2x change in step size. Higher quantization parameter values ​​imply a larger quantization step size and a coarser representation of the quantized coefficients.

[0077] The video encoder 200 and video decoder 300 can be configured to process quantization and scaling matrices. In video decoding, the residual obtained after the prediction operation is transformed using DCT2 or other transform operations. Subsequently, the transform coefficients are quantized, and the quantized coefficients are entropy decoded.

[0078] The quantization process is controlled by two factors: quantization parameters and a scaling matrix. The quantization parameters have been described above. At the decoder (e.g., video decoder 300), a scaling factor corresponding to the quantization parameters is determined. This scaling factor is applied as follows:

[0079] levelScale[][qP%6])<<(qP / 6)

[0080] Where qP is the quantization parameter, and levelScale[][] is an array defined as follows.

[0081] The list levelScale[][] is specified as levelScale[j][k] = {{40,45,51,57,64,72},{57,64,72,80,90,102}}, where j = 0..1 and k = 0..5.

[0082] A difference of 6 in QP results in a bit shift of 1, and thus the scaling associated with QP is applied by shifting (qP / 6) and scaling calculated using qp%6.

[0083] Furthermore, a scaling parameter is applied to each coefficient. The scaling parameter can be different for different coefficients. The scaling factor associated with the scaling matrix is ​​derived as follows:

[0084] –The intermediate scaling factor m[x][y] is derived as follows:

[0085] – m[x][y] is set to 16 if one or more of the following conditions are true:

[0086] –sps_scaling_list_enabled_flag equals 0.

[0087] –transform_skip_flag[xTbY][yTbY] equals 1.

[0088] –Otherwise, the following applies:

[0089] m[x][y] = ScalingFactor[Log2(nTbW)][Log2(nTbH)][matrixId][x][y], where matrixId is specified as shown in Table 7-5.

[0090] (8-958)

[0091] The final scaling factor used in inverse quantization is obtained by multiplying two scaling terms (from QP and the scaling matrix), as follows:

[0092] – If dep_quant_enabled_flag equals 1, the following applies:

[0093] ls[x][y]=(m[x][y]*levelScale[rectNonTsFlag][(qP+1)%6])<<((qP+1) / 6)(8-959)

[0094] – Otherwise (dep_quant_enabled_flag equals 0), the following applies:

[0095] ls[x][y]=(m[x][y]*levelScale[rectNonTsFlag][qP%6])<<(qP / 6) (8-960)

[0096] The scaling transformation coefficients are derived as follows, and the results are then sent to the inverse quantization step.

[0097] The value dnc[x][y] is derived as follows:

[0098] dnc[x][y]=(dz[x][y]*ls[x][y]+bdOffset)>>bdShift (8-963)

[0099] – The scaling transformation coefficients d[x][y] are derived as follows:

[0100] d[x][y]=Clip3(CoeffMin,CoeffMax,dnc[x][y]) (8-964)

[0101] The following describes the signaling and the definition of the scaling matrix. The video encoder 200 and video decoder 300 can be configured to process the scaling matrix. The scaling matrix is ​​a set of coefficients used to scale the transform coefficients. Two uses of the scaling matrix are rate control and perceptual quality control. Rate control of video is typically performed by adjusting the QP value of a block. However, QP differences result in a uniform scaling factor applied to the entire block. The scaling matrix can be used for relative control between various coefficients within a transform block. For example, the scaling matrix can be defined to make low-frequency coefficients less quantized than high-frequency coefficients, which may be beneficial for content with less high-frequency content. For perceptual quality control, the scaling matrix can also be used to control the relative precision of coefficients within a transform block, so that the perceptual quality of the video is maintained at a lower bit rate. Human Visual System (HVS)-based quantization using the scaling matrix can provide better quality video for certain types of content.

[0102] The scaling matrix is ​​signaled using a scaling list that is signaled in the Adaptive Parameter Set (APS). In the SPS, the scaling list can be enabled or disabled. If the SPS indicates that the scaling list is enabled, additional signaling in the slice header can be used to turn the scaling matrix on and off.

[0103] A scaling matrix is ​​defined for each transform block size and prediction type. The matrix is ​​derived from a scaling list. The syntax for the scaling list used for signaling in PPS / SPS is as follows:

[0104]

[0105]

[0106] The semantics of scaling matrices are provided in section 7.4.3.21 of JVET-Q2001 (e.g., VVC draft 8) and are reproduced herein for reference.

[0107] The semantics of the scaled list data are described below.

[0108] A scaling_matrix_for_lfnst_disabled_flag value of 1 specifies that the scaling matrix is ​​not applied to blocks decoded using LFNST. A scaling_matrix_for_lfnst_disabled_flag value of 0 specifies that the scaling matrix can be applied to blocks decoded using LFNST.

[0109] A scaling_list_chroma_present_flag value of 1 indicates that the chroma scaling list exists in scaling_list_data(). A scaling_list_chroma_present_flag value of 0 indicates that the chroma scaling list does not exist in scaling_list_data(). A requirement for bitstream consistency is that scaling_list_chroma_present_flag (if present) should be 0 when ChromaArrayType is 0, and should be 1 when ChromaArrayType is not 0.

[0110] A scaling_list_copy_mode_flag[id] equal to 1 indicates that the value of the scaling list is the same as the value of the reference scaling list. The reference scaling list is specified by scaling_list_pred_id_delta[id]. A scaling_list_copy_mode_flag[id] equal to 0 indicates that scaling_list_pred_mode_flag exists.

[0111] A scaling_list_pred_mode_flag[id] equal to 1 indicates that the value of the scaling list can be predicted from a reference scaling list. The reference scaling list is specified by scaling_list_pred_id_delta[id]. A scaling_list_pred_mode_flag[id] equal to 0 indicates that the value of the scaling list can be explicitly signaled. When it does not exist, the value of scaling_list_pred_mode_flag[id] is inferred to be 0.

[0112] `scaling_list_pred_id_delta[id]` specifies the reference scaling list used to derive the predicted scaling matrix `ScalingMatrixPred[id]`. If it does not exist, the value of `scaling_list_pred_id_delta[id]` is inferred to be 0. The value of `scaling_list_pred_id_delta[id]` should be in the range of 0 to `maxIdDelta`, which is derived from `id`, as follows:

[0113] maxIdDelta=(id<2)? id:((id<8)?(id-2):(id-8))(106)

[0114] The variables refId and matrixSize are derived as follows:

[0115] refId=id-scaling_list_pred_id_delta[id] (107)

[0116] matrixSize=(id<2)? 2:((id<8)?4:8) (108)

[0117] The (matrixSize)×(matrixSize) array ScalingMatrixPred[x][y] (where x = 0..matrixSize-1, y = 0..matrixSize-1) and the variable ScalingMatrixDCPred can be derived as follows:

[0118] - When both scaling_list_copy_mode_flag[id] and scaling_list_pred_mode_flag[id] are equal to 0, all elements of ScalingMatrixPred are set to equal to 8, and the value of ScalingMatrixDCPred is set to equal to 8.

[0119] Otherwise, when scaling_list_pred_id_delta[id] equals 0, all elements of ScalingMatrixPred are set to equal to 16, and ScalingMatrixDCPred is set to equal to 16.

[0120] - Otherwise (scaling_list_copy_mode_flag[id] or scaling_list_pred_mode_flag[id] equals 1 and scaling_list_pred_id_delta[id] is greater than 0), ScalingMatrixPred is set to equal ScalingMatrixRec[refId], the following applies to ScalingMatrixDCPred:

[0121] - If refId is greater than 13, then ScalingMatrixDCPred is set to equal ScalingMatrixDCRec[refId-14].

[0122] - Otherwise (refId is less than or equal to 13), ScalingMatrixDCPred is set to equal ScalingMatrixPred[0][0].

[0123] When id is greater than 13, scaling_list_dc_coef[id-14] is used to deduce the value of the variable ScalingMatrixDC[id-14], as shown below:

[0124] ScalingMatrixDCRec[id-14]=(ScalingMatrixDCPred+scaling_list_dc_coef[id-14])&255 (109)

[0125] If it does not exist, the value of scaling_list_dc_coef[id-14] is inferred to be equal to 0. The value of scaling_list_dc_coef[id–14] should be in the range of -128 to 127 (inclusive). The value of ScalingMatrixDCRec[id-14] should be greater than 0.

[0126] `scaling_list_delta_coef[id][i]` specifies the difference between the current matrix coefficients `ScalingList[id][i]` and the previous matrix coefficients `ScalingList[id][i-1]` when `scaling_list_copy_mode_flag[id]` equals 0. The value of `scaling_list_delta_coef[id][i]` should be in the range of -128 to 127 (inclusive). When `scaling_list_copy_mode_flag[id]` equals 1, all elements of `ScalingList[id]` are set to 0.

[0127] The (matrixSize)×(matrixSize) array ScalingMatrixRec[id] is derived as follows:

[0128] ScalingMatrixRec[id][x][y]=(ScalingMatrixPred[x][y]+ScalingList[id][k])&255 (110)

[0129] Among them, k=0...(matrixSize*matrixSize-1), x=DiagScanOrder[Log2(matrixSize)][Log2(matrixSize)][k][0], and y=DiagScanOrder[Log2(matrixSize)][Log2(matrixSize)][k][1]

[0130] The value of ScalingMatrixRec[id][x][y] should be greater than 0.

[0131] The scaling matrix, represented by the variables ScalingFactor[wId][hId][matrixId][x][y], is derived from the scaling list data. wId and hId refer to the sizeID variables representing the size of the transform block. sizeId and matrixId are given in the following table:

[0132] Table 7-5 – Relationship between Scaling List IDs and Prediction Modes and Components

[0133]

[0134] Some salient features of the scaling matrix and its derivatives are provided below:

[0135] • Specify a scaling matrix for each of the three color components and two prediction types: inter-frame prediction and IBC are treated as one type, and intra-frame prediction is treated as another type.

[0136] • Specify the scaling list (and the resulting matrix) for the square transform block. For a rectangular TB, the scaling matrix is ​​derived from the scaling matrix of the corresponding square TB.

[0137] For 16×16, 32×32, and 64×64 scaling matrices, only 64 coefficients are specified as an 8×8 grid, and larger blocks of matrix coefficients are obtained by upsampling the coefficients to the desired size. In such cases, the DC coefficients are also signaled.

[0138] Based on the size of the zoom list used for signaling, the zoom list is divided into three categories:

[0139] Category 1: Scaling lists with IDs 0 and 1; these lists have a size of 4 (i.e., 2×2) coefficients.

[0140] Category 2: Scaling lists with IDs 2 to 7 (inclusive); these lists have a size of 16 (i.e., 4×4) coefficients.

[0141] Category 3: Scaling lists with IDs from 8 to 27 (inclusive); these lists have a size of 64 (i.e., 8×8) coefficients.

[0142] A total of 28 scaling lists can be specified in the scaling list APS. Within each of the three categories, a scaling list can be predicted or copied from other scaling lists with smaller IDs. For example, a scaling list with ID 5 (category 2) can be predicted from any scaling list with IDs 2 to 4 (inclusive) (also category 2), but not from categories 1 and 3 or scaling lists with IDs 6 and 7. Prediction can be copied (the value of the reference scaling list is used without modification) or incremental (delta) prediction (the incremental value is signaled to the value of the reference scaling matrix). DC coefficients can also be copied or predicted from the DC coefficients of the reference scaling list, or explicitly signaled, when signaling for a specific sizeID.

[0143] Figure 3 This is a block diagram illustrating an example video encoder 200 that can perform the techniques of this disclosure. Figure 3 This disclosure is provided for illustrative purposes and should not be construed as limiting the techniques extensively illustrated and described herein. For purposes of explanation, this disclosure describes a video encoder 200 based on VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265) technologies. However, the techniques of this disclosure can be implemented by video encoding devices configured for other video decoding standards.

[0144] exist Figure 3 In the example, the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filter unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, mode selection unit 202, residual generation unit 204, transform processing unit 206, quantization unit 208, inverse quantization unit 210, inverse transform processing unit 212, reconstruction unit 214, filter unit 216, DPB 218, and entropy coding unit 220 can be implemented in one or more processors or in processing circuitry. For example, units of the video encoder 200 can be implemented as one or more circuit or logic elements as part of hardware circuitry or as part of a processor ASIC of an FPGA. Furthermore, the video encoder 200 may include additional or alternative processors or processing circuitry to perform these and other functions.

[0145] The video data storage device 230 can store video data to be encoded by the components of the video encoder 200. The video encoder 200 can obtain data from, for example, a video source 104 (…). Figure 1The video encoder 200 receives video data stored in video data memory 230. DPB 218 can act as a reference picture memory, storing reference video data for prediction of subsequent video data by the video encoder 200. Video data memory 230 and DPB 218 can be formed from any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Video data memory 230 and DPB 218 can be provided by the same memory device or separate memory devices. In various examples, video data memory 230 can be on-chip (as shown) with other components of the video encoder 200, or off-chip relative to those components.

[0146] In this disclosure, references to video data memory 230 should not be construed as being limited to memory within video encoder 200 (unless specifically described as such) or to memory outside video encoder 200 (unless specifically described as such). Rather, references to video data memory 230 should be understood as a reference memory storing video data (e.g., video data of the current block to be encoded) received by video encoder 200 for encoding. Figure 1 The memory 106 can also provide temporary storage for the outputs from the various units of the video encoder 200.

[0147] Show Figure 3 The various units help to understand the operations performed by the video encoder 200. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit refers to a circuit that provides specific functionality and is preset to perform certain operations. A programmable circuit refers to a circuit that can be programmed to perform various tasks and provides flexible functionality in the operations it can perform. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of that software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally constant. In some examples, one or more units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more units can be integrated circuits.

[0148] The video encoder 200 may include an arithmetic logic unit (ALU), an essential function unit (EFU), digital circuitry, analog circuitry, and / or a programmable core, all formed by programmable circuitry. In an example where the operation of the video encoder 200 is performed using software executed by programmable circuitry, memory 106 ( Figure 1The video encoder 200 may store instructions (e.g., object code) of the software received and executed by the video encoder 200, or such instructions may be stored in another memory (not shown) within the video encoder 200.

[0149] The video data storage unit 230 is configured to store received video data. The video encoder 200 can retrieve images of the video data from the video data storage unit 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data storage unit 230 can be the raw video data to be encoded.

[0150] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra-frame prediction unit 226. The mode selection unit 202 may include additional functional units to perform video prediction based on other prediction modes. As an example, the mode selection unit 202 may include a palette unit, an intra-frame block copying unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0151] Mode selection unit 202 typically coordinates multiple coding passes to test combinations of coding parameters and the resulting rate distortion values ​​for such combinations. Coding parameters may include segmenting the CTU into CUs, the prediction mode for the CUs, the transform type for the residual data of the CUs, and the quantization parameters for the residual data of the CUs. Mode selection unit 202 can ultimately select a combination of coding parameters that has a better rate distortion value than other tested combinations.

[0152] The video encoder 200 can segment an image obtained from the video data storage 230 into a series of CTUs and encapsulate one or more CTUs in a slice. The mode selection unit 202 can segment the CTUs of the image according to a tree structure such as the QTBT structure or quadtree structure of HEVC described above. As described above, the video encoder 200 can form one or more CUs by segmenting CTUs according to the tree structure. Such CUs can also be referred to as "video blocks" or "blocks".

[0153] Generally, mode selection unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra-prediction unit 226) to generate predicted blocks for the current block (e.g., the current CU, or the overlapping portion of PU and TU in HEVC). For inter-frame prediction of the current block, motion estimation unit 222 may perform motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously decoded pictures stored in DPB 218). Specifically, motion estimation unit 222 may calculate values ​​representing how similar a potential reference block is to the current block, for example, based on sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), etc. Motion estimation unit 222 may typically perform these calculations using sample-by-sample differences between the current block and the considered reference blocks. Motion estimation unit 222 may identify reference blocks with the lowest values ​​produced by these calculations, thus indicating the reference block that most closely matches the current block.

[0154] Motion estimation unit 222 can generate one or more motion vectors (MVs) that define the position of a reference block in a reference image relative to the position of a current block in the current image. Motion estimation unit 222 can then provide the motion vectors to motion compensation unit 224. For example, for unidirectional inter-frame prediction, motion estimation unit 222 can provide a single motion vector, while for bidirectional inter-frame prediction, motion estimation unit 222 can provide two motion vectors. Motion compensation unit 224 can then use the motion vectors to generate prediction blocks. For example, motion compensation unit 224 can use the motion vectors to acquire data for the reference blocks. As another example, if the motion vectors have fractional sampling precision, motion compensation unit 224 can interpolate values ​​for the prediction blocks according to one or more interpolation filters. Furthermore, for bidirectional inter-frame prediction, motion compensation unit 224 can acquire data for two reference blocks identified by the corresponding motion vectors and combine the acquired data, for example, by per-sample averaging or weighted averaging.

[0155] As another example, for intra-prediction or intra-prediction decoding, intra-prediction unit 226 can generate a prediction block based on samples adjacent to the current block. For example, in directional mode, intra-prediction unit 226 can typically mathematically combine the values ​​of adjacent samples and fill these calculated values ​​in a defined direction across the current block to generate a prediction block. As another example, in DC mode, intra-prediction unit 226 can calculate the average of the adjacent samples of the current block and generate a prediction block to include this average for each sample in the prediction block.

[0156] Mode selection unit 202 provides the prediction block to residual generation unit 204. Residual generation unit 204 receives the original, uncoded version of the current block from video data memory 230 and the prediction block from mode selection unit 202. Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines the residual block for the current block. In some examples, residual generation unit 204 can also determine the differences between sample values ​​in the residual block to generate a residual block using residual differential pulse decode-modulation (RDPCM). In some examples, one or more subtractor circuits performing binary subtraction can be used to form residual generation unit 204.

[0157] In the example where mode selection unit 202 divides a CU into PUs, each PU can be associated with a luma prediction unit and a corresponding chroma prediction unit. Video encoder 200 and video decoder 300 can support PUs of various sizes. As indicated above, the size of a CU can refer to the size of its luma decoding block, and the size of a PU can refer to the size of its luma prediction unit. Assuming a particular CU is 2N×2N, video encoder 200 can support PU sizes of 2N×2N or N×N for intra-frame prediction, and symmetrical PU sizes of 2N×2N, 2N×N, N×2N, N×N, or similar sizes for inter-frame prediction. Video encoder 200 and video decoder 300 can also support asymmetric segmentation of PU sizes of 2N×nU, 2N×nD, nL×2N, and nR×2N for inter-frame prediction.

[0158] In the example where mode selection unit 202 does not further divide the CU into PUs, each CU can be associated with a luminance decoding block and a corresponding chrominance decoding block. As mentioned above, the size of the CU can refer to the size of the luminance decoding block of the CU. The video encoder 200 and the video decoder 300 can support CU sizes of 2N×2N, 2N×N, or N×2N.

[0159] For other video decoding techniques such as intra-block copy mode decoding, affine mode decoding, and linear model (LM) mode decoding (to name just a few), mode selection unit 202 generates a prediction block for the current block being encoded via various units associated with the decoding technique. In some examples, such as palette mode decoding, mode selection unit 202 may not generate a prediction block, but instead generate syntax elements indicating how the block should be reconstructed based on the selected palette. In this mode, mode selection unit 202 may provide these syntax elements to entropy coding unit 220 for encoding.

[0160] As described above, the residual generation unit 204 receives video data for the current block and the corresponding prediction block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0161] Transform processing unit 206 applies one or more transformations to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). Transform processing unit 206 may apply various transformations to the residual block to form the transform coefficient block. For example, transform processing unit 206 may apply a Discrete Cosine Transform (DCT), direction transformation, Karhunen-Loeve Transform (KLT), or conceptually similar transformations to the residual block. In some examples, transform processing unit 206 may perform multiple transformations on the residual block, such as primary and secondary transformations (e.g., rotation transformations). In some examples, transform processing unit 206 does not apply any transformations to the residual block.

[0162] Quantization unit 208 can quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. Quantization unit 208 can quantize the transform coefficients of the transform coefficient block based on the quantization parameter (QP) value associated with the current block. Video encoder 200 (e.g., via mode selection unit 202) can adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce information loss; therefore, the quantized transform coefficients may have lower precision than the original transform coefficients generated by transform processing unit 206.

[0163] The inverse quantization unit 210 and the inverse transform processing unit 212 can apply inverse quantization and inverse transform to the quantized transform coefficient block, respectively, to reconstruct the residual block based on the transform coefficient block. The reconstruction unit 214 can generate a reconstructed block corresponding to the current block based on the reconstructed residual block and the prediction block generated by the mode selection unit 202 (although potentially with some degree of distortion). For example, the reconstruction unit 214 can add samples of the reconstructed residual block to the corresponding samples of the prediction block generated by the mode selection unit 202 to generate the reconstructed block.

[0164] Filter unit 216 can perform one or more filtering operations on the reconstructed block. For example, filter unit 216 can perform a deblocking operation to reduce blocky artifacts along the edges of the CU. In some examples, the operation of filter unit 216 can be skipped.

[0165] The video encoder 200 stores reconstructed blocks in the DPB 218. For example, in an example where the operation of the filter unit 216 is not required, the reconstruction unit 214 can store the reconstructed blocks in the DPB 218. In an example where the operation of the filter unit 216 is required, the filter unit 216 can store the filtered reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can obtain a reference picture formed by the reconstructed (and possibly filtered) blocks from the DPB 218 to perform inter-frame prediction of blocks in the subsequently encoded picture. In addition, the intra-frame prediction unit 226 can use the reconstructed blocks in the DPB 218 of the current picture to perform intra-frame prediction of other blocks in the current picture.

[0166] Generally, entropy coding unit 220 can entropy code syntax elements received from other functional components of video encoder 200. For example, entropy coding unit 220 can entropy code quantized transform coefficient blocks from quantization unit 208. As another example, entropy coding unit 220 can entropy code predictive syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from mode selection unit 202. Entropy coding unit 220 can perform one or more entropy coding operations on syntax elements, another example of video data, to generate entropy-coded data. For example, entropy coding unit 220 can perform context-adaptive variable-length decoding (CAVLC), CABAC, variable-to-variable (V2V) length decoding, syntax-based context-adaptive binary arithmetic decoding (SBAC), probabilistic interval partitioned entropy (PIPE) decoding, exponential-Golomb coding, or another type of entropy coding operation on the data. In some examples, the entropy coding unit 220 can operate in a bypass mode where syntactic elements are not entropy encoded.

[0167] The video encoder 200 can output a bitstream containing the entropy-encoded syntax elements required to reconstruct slices or blocks of images. Specifically, the entropy coding unit 220 can output this bitstream.

[0168] The operations described above are relative to blocks. Such descriptions should be understood as operations applied to luma decoding blocks and / or chroma decoding blocks. As described above, in some examples, the luma decoding block and chroma decoding block are the luma and chroma components of the CU. In some examples, the luma decoding block and chroma decoding block are the luma and chroma components of the PU.

[0169] In some examples, the operations performed relative to the chroma decoding block do not need to be repeated for the luma decoding block. As an example, the operations used to identify the motion vector (MV) and reference image for the luma decoding block do not need to be repeated for identifying the MV and reference image for the chroma block. Instead, the MV for the luma decoding block can be scaled to determine the MV for the chroma block, and the reference image can be the same. As another example, the intra-frame prediction processing can be the same for both the luma and chroma decoding blocks.

[0170] Video encoder 200 represents an example of a device configured to encode video data, the device including a memory configured to store the video data, and one or more processing units implemented in circuitry and configured to perform the following operations: obtaining a scaling list for adaptive color transformation (ACT); encoding syntax elements of a set of block types to which the specified scaling list can be applied in the decoded video bitstream; and applying ACT to video data blocks having block types included in the set of block types.

[0171] Video encoder 200 represents an example of a device configured to encode video data, the device including a memory configured to store the video data, and one or more processing units implemented in circuitry and configured to perform the following operations: obtaining a scaling list of Low Frequency Inseparable Transform (LFNST); encoding syntax elements of a set of block types to which the specified scaling list can be applied in the decoded video bitstream; and applying the LFNST to video data blocks having block types included in the set of block types.

[0172] Figure 4 This is a block diagram illustrating an example video decoder 300 that can perform the techniques disclosed herein. Figure 4 This disclosure is provided for illustrative purposes and does not limit the techniques extensively illustrated and described herein. For illustrative purposes, this disclosure describes a video decoder 300 based on VVC (ITU-T H.266, under development) and HEVC (ITU-T H.265) technologies. However, the techniques of this disclosure can be implemented by video decoding devices configured for other video decoding standards.

[0173] exist Figure 4In the example, the video decoder 300 includes a decoded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filter unit 312, and a decoded picture buffer (DPB) 314. Any one or all of the CPB memory 320, entropy decoding unit 302, prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, filter unit 312, and DPB 314 can be implemented in one or more processors or processing circuitry. For example, units of the video decoder 300 can be implemented as one or more circuit or logic elements as part of hardware circuitry or as part of a processor ASIC of an FPGA. Furthermore, the video decoder 300 may include additional or alternative processors or processing circuitry to perform these or other functions.

[0174] The prediction processing unit 304 includes a motion compensation unit 316 and an intra-prediction unit 318. The prediction processing unit 304 may include additional units to perform predictions based on other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block copying unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, etc. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0175] CPB memory 320 can store video data to be decoded by components of video decoder 300, such as encoded video bitstreams. The video data stored in CPB memory 320 can be, for example, from computer-readable medium 110 (…). Figure 1The CPB memory 320 may include a CPB that stores encoded video data (e.g., syntax elements) from the encoded video bitstream. Additionally, the CPB memory 320 may store video data other than the syntax elements of the decoded pictures, such as temporary data representing the output from various units of the video decoder 300. The DPB 314 typically stores decoded pictures that the video decoder 300 may output and / or that the video decoder 300 may use as reference video data when decoding subsequent data or pictures from the encoded video bitstream. The CPB memory 320 and DPB 314 may be formed from any of a variety of memory devices, such as DRAM, including SDRAM, MRAM, RRAM, or other types of memory devices. The CPB memory 320 and DPB 314 may be provided by the same memory device or separate memory devices. In various examples, the CPB memory 320 may be located on-chip with other components of the video decoder 300 or off-chip relative to those components.

[0176] Additionally or alternatively, in some examples, the video decoder 300 can be drawn from the memory 120 ( Figure 1 The decoded video data is obtained. That is, memory 120 can store data together with CPB memory 320 as discussed above. Similarly, when some or all of the functionality of video decoder 300 is implemented in software to be executed by the processing circuitry of video decoder 300, memory 120 can store instructions to be executed by video decoder 300.

[0177] Figure 4 The various units shown are illustrated to aid in understanding the operations performed by the video decoder 300. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Figure 3 Similarly, a fixed-function circuit refers to a circuit that provides a specific function and is pre-programmed to perform certain operations. A programmable circuit, on the other hand, is a circuit that can be programmed to perform various tasks and provides flexible functionality within the operable operations. For example, a programmable circuit can execute software or firmware that causes it to operate in a manner defined by instructions from that software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is typically constant. In some examples, one or more units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0178] The video decoder 300 may include an ALU, an EFU, digital circuitry, analog circuitry, and / or a programmable core formed by programmable circuitry. In an example where the operation of the video decoder 300 is performed by software executed on the programmable circuitry, on-chip or off-chip memory may store instructions (e.g., object code) of the software received and executed by the video decoder 300.

[0179] Entropy decoding unit 302 can receive encoded video data from the CPB and perform entropy decoding on the video data to reproduce the syntax elements. Prediction processing unit 304, inverse quantization unit 306, inverse transform processing unit 308, reconstruction unit 310, and filter unit 312 can generate decoded video data based on the syntax elements extracted from the bitstream.

[0180] Typically, the video decoder 300 reconstructs the image on a block-by-block basis. The video decoder 300 can perform the reconstruction operation on each block individually (where the block currently being reconstructed, i.e. the block being decoded, can be referred to as the "current block").

[0181] Entropy decoding unit 302 can entropy decode the syntax elements of the quantized transform coefficients that define the quantized transform coefficient block, as well as transform information such as quantization parameters (QP) and / or transform mode indications. Inverse quantization unit 306 can use the QP associated with the quantized transform coefficient block to determine the degree of quantization, and similarly, determine the degree of inverse quantization to be applied by inverse quantization unit 306. Inverse quantization unit 306 can, for example, perform a bit-by-bit left shift operation to inverse quantize the quantized transform coefficients. Inverse quantization unit 306 can thus form a transform coefficient block including the transform coefficients.

[0182] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 can apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 can apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotation transform, an inverse direction transform, or another inverse transform to the transform coefficient block.

[0183] Furthermore, prediction processing unit 304 generates prediction blocks based on prediction information syntax elements entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax elements indicate that the current block is inter-frame predicted, motion compensation unit 316 can generate prediction blocks. In this case, the prediction information syntax elements may indicate a reference picture from which the reference block is to be obtained in DPB 314, and a motion vector identifying the position of the reference block in the reference picture relative to the current block in the current picture. Motion compensation unit 316 can typically be configured in a manner substantially similar to that of motion compensation unit 224 ( Figure 3The method described herein is used to perform inter-frame prediction processing.

[0184] As another example, if the prediction information syntax element indicates that the current block is intra-predictive, then intra-predictive unit 318 can generate a prediction block according to the intra-predictive mode indicated by the prediction information syntax element. Similarly, intra-predictive unit 318 can generally be configured in a manner substantially similar to that of intra-predictive unit 226. Figure 3 Intra-prediction processing is performed in the manner described. Intra-prediction unit 318 can obtain data of neighboring samples of the current block from DPB 314.

[0185] Reconstruction unit 310 can use the prediction block and the residual block to reconstruct the current block. For example, reconstruction unit 310 can add the samples of the residual block to the corresponding samples of the prediction block to reconstruct the current block.

[0186] Filter unit 312 can perform one or more filtering operations on the reconstructed block. For example, filter unit 312 can perform a deblocking operation to reduce blocky artifacts along the edges of the reconstructed block. The operation of filter unit 312 does not necessarily have to be performed in all examples.

[0187] The video decoder 300 can store reconstructed blocks in the DPB 314. For example, in an example where the filter unit 312 is not operated, the reconstruction unit 310 can store the reconstructed blocks in the DPB 314. In an example where the filter unit 312 is operated, the filter unit 312 can store the filtered reconstructed blocks in the DPB 314. As discussed above, the DPB 314 can provide reference information to the prediction processing unit 304, such as samples of the current image for intra-frame prediction and samples of previously decoded images for subsequent motion compensation. Furthermore, the video decoder 300 can output decoded images (e.g., decoded video) from the DPB 314 for subsequent processing, such as... Figure 1 The display is presented on the display device 118.

[0188] In this way, video decoder 300 represents an example of a video decoding device, which includes a memory configured to store video data, and one or more processing units implemented in circuitry and configured to perform the following operations: obtaining a scaling list for adaptive color transformation (ACT); decoding syntax elements of a block type set to which the specified scaling list can be applied from the decoded video bitstream; and applying ACT to video data blocks having block types included in the block type set.

[0189] Video decoder 300 represents an example of a device configured to decode video data, the device including a memory configured to store the video data, and one or more processing units implemented in a circuit and configured to perform the following operations: obtaining a scaling list of low-frequency non-separable transforms (LFNST); decoding syntax elements of a block type set to which the specified scaling list can be applied from the decoded video bitstream; and applying the LFNST to video data blocks having block types included in the block type set.

[0190] Figure 5 This is a flowchart illustrating an example method for encoding the current block. The current block may include the current CU. Although regarding video encoder 200 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform similar actions. Figure 5 The method of the method.

[0191] In this example, the video encoder 200 initially predicts the current block (350). For example, the video encoder 200 may form a predicted block for the current block. Then, the video encoder 200 may compute a residual block for the current block (352). To compute the residual block, the video encoder 200 may compute the difference between the original uncoded block and the predicted block for the current block. The video encoder 200 may then transform the residual block and quantize the transform coefficients of the residual block (354). Next, the video encoder 200 may scan the quantized transform coefficients of the residual block (356). During or after the scan, the video encoder 200 may entropy encode the transform coefficients (358). For example, the video encoder 200 may encode the transform coefficients using CAVLC or CABAC. The video encoder 200 may then output the entropy-encoded data of the block (360).

[0192] Figure 6 This is a flowchart illustrating an example method for decoding the current block of video data. The current block may include the current CU. Although regarding video decoder 300 ( Figure 1 and Figure 4 The description is provided, but it should be understood that other devices can be configured to perform similar actions. Figure 6 The method of the method.

[0193] The video decoder 300 can receive entropy-encoded data of the current block, such as entropy-encoded prediction information and entropy-encoded data of the transform coefficients of the residual block corresponding to the current block (370). The video decoder 300 can entropy decode the entropy-encoded data to determine the prediction information of the current block and the transform coefficients of the reconstructed residual block (372). The video decoder 300 can predict the current block, for example, using an intra-frame or inter-frame prediction mode indicated by the prediction information of the current block (374), to compute the prediction block of the current block. The video decoder 300 can then inversely scan the reconstructed transform coefficients (376) to create a block of quantized transform coefficients. The video decoder 300 can then inversely quantize the transform coefficients and apply the inverse transform to the transform coefficients to produce the residual block (378). The video decoder 300 can finally decode the current block by combining the prediction block and the residual block (380).

[0194] Figure 7 This is a flowchart illustrating an example method for adaptive scaling list control according to one or more techniques of this disclosure. Although regarding video decoder 300 ( Figure 1 and Figure 4 The description is provided, but it should be understood that other devices can be configured to perform similar actions. Figure 7 The method. For example, video encoder 200 ( Figure 1 and Figure 3 It can be configured to perform similar actions. Figure 7 The method of the method.

[0195] The video decoder 300 can decode an explicitly defined scaling list via the decoded video bitstream (702). For example, the entropy decoding unit 302 can decode one or more syntax elements representing the scaling list (e.g., scaling_list_pred_id_delta, scaling_list_dc_coef, scaling_list_delta_coef, etc.).

[0196] The video decoder 300 can determine the set of block types to which an explicitly defined scaling list is eligible (704) based on one or more syntax elements in the Sequence Parameter Set (SPS) of the decoded video bitstream. For example, the entropy decoding unit 302 can decode one or more syntax elements (e.g., ScalingListType) and determine the block type set based on the values ​​of the syntax elements. In some examples, the block type set may include ACT block types or other blocks (e.g., non-ACT block types). If an adaptive color transformation is applied to a block, that block may be an ACT block type.

[0197] In some examples, one or more syntax elements may include a first syntax element with a value specifying whether ACT is enabled. If the value of the first syntax element specifies that ACT is not enabled, the video decoder 300 can determine that the block type set does not include the ACT block type. One or more syntax elements may include a second syntax element with a value specifying whether the block type set includes blocks to which color space conversion has been applied (e.g., `sps_scaling_matrix_for_alternative_colour_space_disabled_flag`). Therefore, the video decoder 300 can determine the set of block types to which the explicitly defined scaling list is eligible based on the second syntax element with a value specifying whether the block type set includes blocks to which color space conversion has been applied.

[0198] The video decoder 300 can apply an explicitly defined scaling list to blocks (706) included in the block type set. For example, the inverse quantization unit 306 can use an explicitly defined scaling list to scale the transform coefficients.

[0199] The following numbered clauses may describe one or more aspects of this disclosure:

[0200] Clause 1A. A method for decoding video data, the method comprising: obtaining a scaling list for adaptive color transformation (ACT); decoding syntax elements of a block type set to which the scaling list can be applied via a decoded video bitstream; and applying the ACT to video data blocks having block types included in the block type set.

[0201] Clause 2A. The method of Clause 1A, wherein the syntax element includes the ScalingListType syntax element.

[0202] Clause 3A. The method according to Clause 2A also includes: responding to a syntax element with a first value to determine that the block type set includes only the ACT block type.

[0203] Clause 4A. The method according to Clause 2A or 3A further includes: in response to a syntactic element having a second value, determining that the block type set includes the ACT block type.

[0204] Clause 5A. The method pursuant to any of Clauses 1A to 4A, wherein applying ACT to the block includes applying ACT to multiple blocks having block types included in the block type set.

[0205] Clause 6A. The method of any of Clauses 1A to 5A, wherein decoding of syntax elements includes decoding syntax elements in a Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), Slice Header (SH), or Adaptive Parameter Set (APS).

[0206] Clause 7A. A method comprising: obtaining a scaling list for low-frequency non-separable transform (LFNST); decoding syntax elements of a block type set to which the scaling list can be applied via a decoded video bitstream; and applying the LFNST to video data blocks having block types included in the block type set.

[0207] Clause 8A. The method of Clause 7A, wherein the syntax element includes the ScalingListType syntax element.

[0208] Clause 9A. The method pursuant to Clause 8A further includes: responding to a syntax element having a first value to determine that the block type set includes only the Adaptive Color Transformation (ACT) block type.

[0209] Clause 10A. The method according to Clause 8A or 9A further includes: in response to a syntax element having a second value, determining that the block type set includes only the LFNST block type.

[0210] Clause 11A. The method pursuant to Clauses 8A to 10A further includes: in response to a syntax element having a third value, determining that the block type set includes the Adaptive Color Transformation (ACT) block type and does not include the LFNST block type.

[0211] Clause 12A. The method pursuant to any of Clauses 7A to 11A, wherein applying LFNST to the block comprises applying LFNST to multiple blocks having block types included in the block type set.

[0212] Clause 13A. The method of any of Clauses 7A to 12A, wherein decoding of syntax elements includes decoding syntax elements in a Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Picture Header (PH), Slice Header (SH), or Adaptive Parameter Set (APS).

[0213] Clause 14A. The method pursuant to any of Clauses 1A through 13A, wherein decoding includes decoding.

[0214] Clause 15A. The method pursuant to any of Clauses 1A through 14A, wherein decoding includes encoding.

[0215] Clause 16A. An apparatus for decoding video data, the apparatus comprising one or more components for performing the method of any one of Clauses 1A to 15A.

[0216] Clause 17A. The device of claim 16A, wherein one or more components include one or more processors implemented in the circuit.

[0217] Clause 18A. Devices pursuant to any of Clauses 16A and 17A may also include a memory for storing video data.

[0218] Clause 19A. An apparatus pursuant to any of Clauses 16A to 18A also includes a display configured to display decoded video data.

[0219] Clause 20A. Devices pursuant to any of Clauses 16A through 19A, wherein the device includes one or more of a camera, computer, mobile device, broadcast receiver device, or set-top box.

[0220] Clause 21A. An apparatus pursuant to any of Clauses 16A to 20A, wherein the apparatus includes a video decoder.

[0221] Clause 22A. Devices pursuant to any of Clauses 16A to 21A, wherein the device includes a video encoder.

[0222] Clause 23A. A computer-readable storage medium having instructions stored thereon, which, when executed, cause one or more processors to perform the method of any one of Clauses 1A to 15A.

[0223] Clause 1B. A method for decoding video data, the method comprising: decoding an explicitly defined scaling list from a decoded video bitstream; determining a set of block types to which the explicitly defined scaling list is eligible to be applied, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and applying the explicitly defined scaling list to blocks included in the block type set.

[0224] Clause 2B. The method pursuant to Clause 1B also includes: not applying an explicitly defined scaling list to blocks not included in the block type set.

[0225] Clause 3B. The method according to Clause 1B, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0226] Clause 4B. According to the method of Clause 3B, one or more syntax elements include a first syntax element having a value specifying whether the ACT is enabled, wherein determining the block type set includes determining that the block type set does not include ACT block types where the ACT is not enabled.

[0227] Clause 5B. According to the method of Clause 4B, one or more syntax elements include a second syntax element having a value indicating whether the specified set of block types includes blocks to which a color space conversion has been applied.

[0228] Clause 6B. An apparatus for decoding video data, the apparatus comprising: a memory configured to store at least a portion of a decoded video bitstream; and one or more processors implemented in circuitry and configured to: decode an explicitly defined scaling list from the decoded video bitstream; determine a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and apply the explicitly defined scaling list to blocks included in the block type set.

[0229] Clause 7B. A device pursuant to Clause 6B, wherein one or more processors are configured to: not apply an explicitly defined scaling list to blocks not included in the block type set.

[0230] Clause 8B. Devices pursuant to Clause 6B, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0231] Clause 9B. A device pursuant to Clause 8B, wherein one or more syntax elements include a first syntax element having a value specifying whether an ACT is enabled, wherein, in order to determine a set of block types, one or more processors are configured to determine that the set of block types does not include ACT block types for which an ACT is not enabled.

[0232] Clause 10B. A device pursuant to Clause 9B, wherein one or more syntax elements include a second syntax element having a value specifying whether a set of block types includes blocks to which a color space conversion has been applied.

[0233] Clause 11B. A method for encoding video data, the method comprising: encoding an explicitly defined scaling list in a decoded video bitstream; encoding one or more syntax elements of a block type set to which the explicitly defined scaling list is eligible in a sequence parameter set (SPS) of the decoded video bitstream; and applying the explicitly defined scaling list to blocks included in the block type set.

[0234] Clause 12B. The method pursuant to Clause 11B also includes: not applying an explicitly defined scaling list to blocks not included in the block type set.

[0235] Clause 13B. The method according to Clause 11B, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0236] Clause 14B. The method of Clause 13B, wherein one or more syntax elements include a first syntax element having a value specifying whether the ACT is enabled, wherein determining the block type set includes determining that the block type set does not include ACT block types where the ACT is not enabled.

[0237] Clause 15B. According to the method of Clause 14B, one or more syntax elements include a second syntax element having a value indicating whether the specified block type set includes blocks to which a color space conversion has been applied.

[0238] Clause 16B. An apparatus for encoding video data, the apparatus comprising: a memory configured to store at least a portion of a decoded video bitstream; and one or more processors implemented in circuitry and configured to: encode an explicitly defined scaling list in the decoded video bitstream; encode one or more syntax elements in a sequence parameter set (SPS) of the decoded video bitstream to which the explicitly defined scaling list is eligible; and apply the explicitly defined scaling list to blocks included in the block type set.

[0239] Clause 17B. A device pursuant to Clause 16B, wherein one or more processors are configured to: not apply an explicitly defined scaling list to blocks not included in the block type set.

[0240] Clause 18B. A device pursuant to Clause 16B, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0241] Clause 19B. A device pursuant to Clause 18B, wherein one or more syntax elements include a first syntax element having a value specifying whether ACT is enabled, wherein, in order to determine a set of block types, one or more processors are configured to determine that the set of block types does not include ACT block types for which ACT is not enabled.

[0242] Clause 20B. A device pursuant to Clause 19B, wherein one or more syntax elements include a second syntax element having a value specifying whether a set of block types includes blocks to which a color space conversion has been applied.

[0243] Clause 21B. An apparatus for decoding video data, the apparatus comprising: means for decoding an explicitly defined scaling list from a decoded video bitstream; means for determining a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and means for applying the explicitly defined scaling list to blocks included in the block type set.

[0244] Clause 22B. An apparatus for encoding video data, the apparatus comprising: means for encoding an explicitly defined scaling list in a decoded video bitstream; means for encoding one or more syntax elements of a block type set to which the explicitly defined scaling list is eligible in a sequence parameter set (SPS) of the decoded video bitstream; and means for applying the explicitly defined scaling list to blocks included in the block type set.

[0245] Clause 23B. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to: decode an explicitly defined scaling list from a decoded video bitstream; determine a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and apply the explicitly defined scaling list to blocks included in the block type set.

[0246] Clause 24B. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to: encode an explicitly defined scaling list in a decoded video bitstream; encode one or more syntax elements in a sequence parameter set (SPS) of the decoded video bitstream to which the explicitly defined scaling list is eligible; and apply the explicitly defined scaling list to blocks included in the block type set.

[0247] Clause 1C. A method for decoding video data, the method comprising: decoding an explicitly defined scaling list from a decoded video bitstream; determining a set of block types to which the explicitly defined scaling list is eligible to be applied, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and applying the explicitly defined scaling list to blocks included in the block type set.

[0248] Clause 2C. The method pursuant to Clause 1C also includes: not applying an explicitly defined scaling list to blocks not included in the block type set.

[0249] Clause 3C. The method according to Clause 1C or 2C, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0250] Clause 4C. The method according to Clause 3C, wherein one or more syntax elements include a first syntax element having a value specifying whether the ACT is enabled, wherein determining the block type set includes determining that the block type set does not include ACT block types where the ACT is not enabled.

[0251] Clause 5C. According to the method of Clause 4C, one or more syntax elements include a second syntax element having a value indicating whether the set of specified block types includes blocks to which a color space conversion has been applied.

[0252] Clause 6C. An apparatus for decoding video data, the apparatus comprising: a memory configured to store at least a portion of a decoded video bitstream; and one or more processors implemented in circuitry and configured to: decode an explicitly defined scaling list from the decoded video bitstream; determine a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and apply the explicitly defined scaling list to blocks included in the block type set.

[0253] Clause 7C. A device pursuant to Clause 6C, wherein one or more processors are configured to: not apply an explicitly defined scaling list to blocks not included in the block type set.

[0254] Clause 8C. Devices pursuant to Clause 6C or 7C, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0255] Clause 9C. A device pursuant to Clause 8C, wherein one or more syntax elements include a first syntax element having a value specifying whether an ACT is enabled, wherein, in order to determine a set of block types, one or more processors are configured to determine that the set of block types does not include ACT block types for which an ACT is not enabled.

[0256] Clause 10C. A device pursuant to Clause 9C, wherein one or more syntax elements include a second syntax element having a value specifying whether a set of block types includes blocks to which a color space conversion has been applied.

[0257] Clause 11C. A method for encoding video data, the method comprising: encoding an explicitly defined scaling list in a decoded video bitstream; encoding one or more syntax elements of a block type set to which the explicitly defined scaling list is eligible in a sequence parameter set (SPS) of the decoded video bitstream; and applying the explicitly defined scaling list to blocks included in the block type set.

[0258] Clause 12C. The method pursuant to Clause 11C also includes: not applying an explicitly defined scaling list to blocks not included in the block type set.

[0259] Clause 13C. The method according to Clause 11C or 12C, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0260] Clause 14C. The method of Clause 13C, wherein one or more syntax elements include a first syntax element having a value specifying whether the ACT is enabled, wherein determining the block type set includes determining that the block type set does not include ACT block types where the ACT is not enabled.

[0261] Clause 15C. According to the method of Clause 14C, one or more syntax elements include a second syntax element having a value indicating whether the specified set of block types includes blocks to which a color space conversion has been applied.

[0262] Clause 16C. An apparatus for encoding video data, the apparatus comprising: a memory configured to store at least a portion of a decoded video bitstream; and one or more processors implemented in circuitry and configured to: encode an explicitly defined scaling list in the decoded video bitstream; encode one or more syntax elements in a sequence parameter set (SPS) of the decoded video bitstream to which the explicitly defined scaling list is eligible; and apply the explicitly defined scaling list to blocks included in the block type set.

[0263] Clause 17C. A device pursuant to Clause 16C, wherein one or more processors are configured to: not apply an explicitly defined scaling list to blocks not included in the block type set.

[0264] Clause 18C. Devices pursuant to Clause 16C or 17C, wherein determining the block type set includes determining whether the block type set includes the Adaptive Color Transformation (ACT) block type.

[0265] Clause 19C. A device pursuant to Clause 18C, wherein one or more syntax elements include a first syntax element having a value specifying whether an ACT is enabled, wherein, in order to determine a set of block types, one or more processors are configured to determine that the set of block types does not include ACT block types for which an ACT is not enabled.

[0266] Clause 20C. A device pursuant to Clause 19C, wherein one or more syntax elements include a second syntax element having a value specifying whether a set of block types includes blocks to which a color space conversion has been applied.

[0267] Clause 21C. An apparatus for decoding video data, the apparatus comprising: means for decoding an explicitly defined scaling list from a decoded video bitstream; means for determining a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and means for applying the explicitly defined scaling list to blocks included in the block type set.

[0268] Clause 22C. An apparatus for encoding video data, the apparatus comprising: means for encoding an explicitly defined scaling list in a decoded video bitstream; means for encoding one or more syntax elements of a block type set to which the explicitly defined scaling list is eligible in a sequence parameter set (SPS) of the decoded video bitstream; and means for applying the explicitly defined scaling list to blocks included in the block type set.

[0269] Clause 23C. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to: decode an explicitly defined scaling list from a decoded video bitstream; determine a set of block types to which the explicitly defined scaling list is eligible, based on the values ​​of one or more syntax elements decoded from a sequence parameter set (SPS) of the decoded video bitstream; and apply the explicitly defined scaling list to blocks included in the block type set.

[0270] Clause 24C. A computer-readable storage medium storing instructions that, when executed, cause one or more processors to: encode an explicitly defined scaling list in a decoded video bitstream; encode one or more syntax elements in a sequence parameter set (SPS) of the decoded video bitstream to which the explicitly defined scaling list is eligible; and apply the explicitly defined scaling list to blocks included in the block type set.

[0271] Clause 1D. Any combination of Clauses 1A to 24C.

[0272] It should be recognized that, depending on the example, certain actions or events of any technique described herein may be performed in a different order, and may be added together, combined, or omitted (e.g., not all described actions or events are necessary for the practice of the technique). Furthermore, in some examples, actions or events may be performed concurrently, for example, through multithreaded processing, interrupt handling, or multiple processors, rather than sequentially.

[0273] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, these functions may be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or a communication medium that includes, for example, any medium facilitating the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory, tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to obtain instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0274] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient tangible storage media. Disks and optical discs as used herein include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0275] Instructions can be executed by one or more processors such as: one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the terms "processor" and "processing circuit" can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0276] The techniques disclosed herein can be implemented in a variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. Rather, as described above, various units can be combined in a codec hardware unit, or provided by a collection of interoperable hardware units including one or more processors as described above, combined with appropriate software and / or firmware.

[0277] Various examples have been described. These and other examples are within the scope of the appended claims.

Claims

1. A method for decoding video data, the method comprising: Decode the explicitly defined scaling list from the decoded video bitstream; Decode one or more syntax elements from the Sequence Parameter Set (SPS) of the decoded video bitstream, the one or more syntax elements indicating whether the explicitly defined scaling list should be applied only to blocks decoded using Adaptive Color Transform (ACT); and Based on the value of one or more of the syntax elements: Apply the explicitly defined scaling list to the block using ACT decoding; as well as The explicitly defined scaling list is not applied to blocks that are not using ACT decoding.

2. The method of claim 1, wherein the one or more syntax elements are tags.

3. An apparatus for decoding video data, the apparatus comprising: At least one memory including instructions; as well as At least one processor is configured to execute the instructions to cause the device to: Decode the explicitly defined scaling list from the decoded video bitstream; Decode one or more syntax elements from the Sequence Parameter Set (SPS) of the decoded video bitstream, the one or more syntax elements indicating whether the explicitly defined scaling list should be applied only to blocks decoded using Adaptive Color Transform (ACT); and Based on the value of one or more of the syntax elements: Apply the explicitly defined scaling list to the block using ACT decoding; as well as The explicitly defined scaling list is not applied to blocks that are not using ACT decoding.

4. The device of claim 3, wherein the one or more syntax elements are tags.

5. A method for encoding video data, the method comprising: Encode the explicitly defined scaling list in the decoded video bitstream; One or more syntax elements are encoded in the Sequence Parameter Set (SPS) of the decoded video bitstream, the one or more syntax elements indicating whether the explicitly defined scaling list is applied only to blocks decoded using Adaptive Color Transform (ACT). Apply the explicitly defined scaling list to the block using ACT decoding; as well as The explicitly defined scaling list is not applied to blocks that are not using ACT decoding.

6. The method of claim 5, wherein the one or more syntax elements are tags.

7. An apparatus for encoding video data, the apparatus comprising: At least one memory including instructions; as well as At least one processor is configured to execute the instructions to cause the device to: Encode the explicitly defined scaling list in the decoded video bitstream; One or more syntax elements are encoded in the Sequence Parameter Set (SPS) of the decoded video bitstream, the one or more syntax elements indicating whether the explicitly defined scaling list is applied only to blocks decoded using Adaptive Color Transform (ACT). Apply the explicitly defined scaling list to the block using ACT decoding; as well as The explicitly defined scaling list is not applied to blocks that are not using ACT decoding.

8. The device of claim 7, wherein the one or more syntax elements are tags.

9. An apparatus for decoding video data, the apparatus comprising: A component used to decode an explicitly defined scaling list from a decoded video bitstream; A component for decoding one or more syntax elements from the Sequence Parameter Set (SPS) of the decoded video bitstream, the one or more syntax elements indicating whether the explicitly defined scaling list is applied only to blocks decoded using Adaptive Color Transform (ACT). as well as A component for applying the explicitly defined scaling list to blocks using ACT decoding, and not applying the explicitly defined scaling list to blocks not using ACT decoding, based on the values ​​of the one or more syntax elements.

10. The device of claim 9, wherein the one or more syntax elements are flags.

11. An apparatus for encoding video data, the apparatus comprising: A component used to encode an explicitly defined scaling list in a decoded video bitstream; A component for encoding one or more syntax elements in the Sequence Parameter Set (SPS) of the decoded video bitstream, the one or more syntax elements indicating whether the explicitly defined scaling list is applied only to blocks decoded using Adaptive Color Transform (ACT). A component for applying the explicitly defined scaling list to blocks that use ACT decoding and not applying the explicitly defined scaling list to blocks that do not use ACT decoding.

12. The device of claim 11, wherein the one or more syntax elements are flags.

13. A computer-readable storage medium storing instructions, which, when executed, cause one or more processors to perform the method according to claim 1 or 2.

14. A computer-readable storage medium storing instructions, which, when executed, cause one or more processors to perform the method according to claim 5 or 6.

Images