A method, device and medium for processing video data

Abstract

Apparatus, systems and methods are described that relate to management of syntax flags regarding chroma formats in video processing. In one representative aspect, a video decoding method includes: for conversion between a video and a codec representation of the video, determining based on a condition whether syntax elements associated with adaptive loop filtering (ALF) operations of chroma components of the video are included in the codec representation. The method also includes performing the conversion based on the determination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a divisional application of the invention patent application with application date of June 1, 2020, application number 202080033241.X, and invention name “Adaptive loop filtering of chrominance components”. Technical Field

[0003] This patent document relates to video encoding and decoding technology, devices and systems. Background Art

[0004] Currently, efforts are being made to improve the performance of current video codec technologies to provide better compression ratios or to provide methods that allow for lower complexity video encoding and decoding schemes or parallel implementations. Some new video codec tools have recently been proposed by industry experts and are currently being tested to determine their effectiveness. Summary of the invention

[0005] Devices, systems and methods related to digital video codecs are described, and in particular, devices, systems and methods related to the management of syntax flags related to chroma formats are described. The described methods can be applied to existing video codec standards (e.g., High Efficiency Video Codec (HEVC) or Versatile Video Codec) and future video codec standards or video codecs.

[0006] In one representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a video and a codec representation of the video, determining based on a condition whether a syntax element associated with an adaptive loop filtering (ALF) operation of a chroma component of the video is included in the codec representation. The method also includes performing the conversion based on the determination.

[0007] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes, for conversion between a video and a codec representation of the video, determining based on a condition whether a syntax element associated with a codec technology for a chroma component of the video is included in the codec representation. The method also includes performing the conversion based on the determination.

[0008] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a video and a codec representation of the video, based on a condition associated with a chroma format of the video, determining whether a syntax element representing a characteristic of a chroma component of the video is included in the codec representation. The method also includes performing the conversion based on the determination.

[0009] In another representative aspect, the disclosed technology can be used to provide a method of video decoding. The method includes determining based on a rule whether one or more syntax flags applicable to a region of a video are included in a syntax structure of a codec representation of the video. The one or more syntax flags indicate a joint chroma residual encoding step for representing the region in the codec representation. The method also includes generating one or more decoded video blocks for the region by parsing the codec representation based on the presence or absence of the one or more syntax flags in the codec representation.

[0010] In one representative aspect, the disclosed technology can be used to provide a method for video encoding. The method includes conditionally encoding one or more syntax flags applicable to a region of the video in a syntax structure of a codec representation of the video based on a rule. The one or more syntax flags indicate a joint chroma residual encoding step for representing the region in the codec representation.

[0011] In another representative aspect, the disclosed technology can be used to provide a method for video decoding. The method includes determining a chroma format of a region of the video, and determining whether one or more syntax flags applicable to the region of the video are included in a syntax structure of a codec representation of the video based on the chroma format. The one or more syntax flags indicate the use of quantization parameter offsets to represent the region in the codec representation. The method also includes generating one or more decoded video blocks of the video region by parsing the codec representation based on the presence or absence of the one or more syntax flags.

[0012] In another representative aspect, the disclosed technology can be used to provide a method for video decoding. The method includes determining a chroma format for a video region, and determining, based on the chroma format, that one or more syntax flags are absent in a syntax structure of a codec representation of the video, indicating use of a secondary transform applicable to the video region. The secondary transform is applied between a dequantization step and an inverse primary transform. The method also includes generating one or more decoded video blocks for the video region by parsing the codec representation based on the absence of the one or more syntax flags.

[0013] In another representative aspect, the disclosed technology can be used to provide a method for video encoding. The method includes determining a chroma format associated with a region of the video and conditionally encoding one or more syntax flags in a syntax structure of a codec representation of the video based on the chroma format. The one or more syntax flags indicate enabling of a quantization parameter offset for representing the region in the codec representation.

[0014] In another representative aspect, the disclosed technology can be used to provide a method for video encoding. The method includes determining a chroma format of a region of the video, and based on the chroma format, generating a codec representation of the video by encoding the region without including one or more syntax flags indicating use of a secondary transform in a syntax structure of the codec representation of the video. The secondary transform is applied between a forward primary transform and a quantization step.

[0015] In another representative aspect, the disclosed technology can be used to provide a method for video decoding. The method includes determining whether one or more syntax flags applicable to the region of the video are missing in the syntax structure of the codec representation of the video based on the chroma format of the video including the region. The one or more syntax flags indicate the use of a luma-dependent chroma residual scaling codec step for representing the region in the codec representation. The luma-dependent chroma residual scaling codec step includes a scaling process in which a scaling coefficient is derived based on reconstructed luma samples, and the scaling process is applied to the chroma residual. The method also includes generating one or more decoded video blocks of the video region by parsing the codec representation based on the determination.

[0016] In another representative aspect, the disclosed technology can be used to provide a method for video decoding. The method includes determining that one or more syntax flags applicable to the region of the video are not present in a syntax structure of a codec representation of the video based on a chroma format of the video including the region. The one or more syntax flags indicate a number of bits used to represent each pulse code modulation sample value of the chroma component. The method also includes generating one or more decoded video blocks of the video region by parsing the codec representation according to the determination.

[0017] In another representative aspect, the disclosed technology can be used to provide a method for video decoding. The method includes determining whether one or more syntax flags applicable to the region of the video are included in a syntax structure of a codec representation of the video based on a chroma format of the video including the region. The one or more syntax flags are associated with an adaptive loop filter (ALF) time domain prediction step for the chroma component. The method also includes generating one or more decoded video blocks of the video region by parsing the codec representation based on the determination.

[0018] In another representative aspect, the disclosed technology can be used to provide a method for video encoding. The method includes determining a chroma format of a video including a region; and based on the chroma format, generating a codec representation of the video by encoding the region without including one or more syntax flags in a syntax structure of a codec representation of the video. The one or more syntax flags indicate the use of a luma-dependent chroma residual scaling codec step for representing the region in the codec representation. The luma-dependent chroma residual scaling codec step includes a scaling process in which a scaling coefficient is derived based on reconstructed luma samples, and the scaling process is applied to the chroma residual.

[0019] In another representative aspect, the disclosed technology can be used to provide a method of video encoding. The method includes determining a chroma format of a video including a region, and generating a codec representation of the video based on the chroma format by encoding the region without including one or more syntax flags in a syntax structure of the codec representation of the video. The one or more syntax flags indicate a number of bits used to represent each pulse code modulation sample value of a chroma component.

[0020] In another representative aspect, the disclosed technology can be used to provide a method of video encoding. The method includes determining a chroma format of a video including a region, and conditionally encoding one or more syntax flags in a syntax structure of a codec representation of the video based on the chroma format. The one or more syntax flags are associated with an adaptive loop filter (ALF) temporal prediction step of a chroma component.

[0021] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for conversion between a block of video and a bitstream representation of the video, determining a plurality of reduced secondary transform (RST) matrices corresponding to a plurality of chrominance color components of the block. The secondary transform is applicable between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform. The method also includes performing the conversion based on the determination.

[0022] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes: for a conversion between a block of video and a bitstream representation of the video, in the case of applying a joint chroma residual encoding and decoding step to the block, determining that one or more matrix indices for a secondary transform are not present in the bitstream representation. The secondary transform can be applied to the block between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform. The joint chroma residual encoding and decoding step includes determining a joint residual that is an average of residuals associated with chroma components. The method also includes performing the conversion based on the determination.

[0023] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a block of video and a bitstream representation of the video, in the case where a secondary transform is applied to the block between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform, determining that there is no syntax flag in the bitstream representation indicating the use of a joint chroma residual encoding and decoding step. The joint chroma residual encoding and decoding step includes determining a joint residual that is an average of residuals associated with chroma components. The method also includes performing the conversion based on the determination.

[0024] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a block of video and a bitstream representation of the video, in the case where a joint chroma residual encoding and decoding step is applied to the block, determining that there is no syntax flag in the bitstream representation indicating the use of a cross-component linear model encoding and decoding step. The joint chroma residual encoding and decoding step includes determining a joint residual that is an average of residuals associated with chroma components. The method also includes performing the conversion based on the determination.

[0025] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a block of video and a bitstream representation of the video, in the case of applying a cross-component linear model to the block, determining that there is no syntax flag in the bitstream representation indicating the use of a joint chroma residual encoding and decoding step. The joint chroma residual encoding and decoding step includes determining a joint residual that is an average of residuals associated with chroma components. The method also includes performing the conversion based on the determination.

[0026] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a block of video and a bitstream representation of the video, determining a correlation between two chroma residuals used in a joint chroma residual encoding and decoding step based on coefficients of a cross-component linear model encoding and decoding step for the block. The joint chroma residual encoding and decoding step includes determining a joint residual that is an average of residuals associated with the chroma components. The method also includes performing the conversion based on the determination.

[0027] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a block of video and a bitstream representation of the video, determining that a luma-dependent chroma residual scaling codec step is disabled when a joint chroma residual codec step is applied to the block. The luma-dependent chroma residual scaling codec step includes a scaling process in which a scaling coefficient is derived based on reconstructed luma samples, and the scaling process is applied to the chroma residual. The method also includes performing the conversion based on the determination.

[0028] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for conversion between a block of a video and a codec representation of the video, determining whether a syntax flag associated with a block indicating the presence of non-zero coefficients is included in a syntax structure in the codec representation based on a rule related to the codec characteristics of the video. The method also includes performing the conversion based on the determination.

[0029] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes: for a conversion between a block of a video and a codec representation of the video, based on a rule associated with the codec characteristics of the video, determining information about a secondary transform with reduced dimensionality. The secondary transform can be applied between a forward primary transform and a quantization step, or between a dequantization step and an inverse primary transform, and the reduced dimensionality is reduced based on the dimensionality of the block. The method also includes performing the conversion according to the determination.

[0030] In one representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes performing a conversion between a current video block and a bitstream representation of the current video block, wherein the conversion includes selectively enabling or disabling signaling of one or more syntax flags associated with the use of one or more of the following in response to detecting at least one condition: a joint chroma residual coding step, a luma-dependent chroma residual scaling (LDCRS) step, a secondary transform step, a quantization step, or an adaptive loop filter step.

[0031] In another representative aspect, the disclosed technology can be used to provide a method of video processing. The method includes performing a conversion between a current video block and a bitstream representation of the current video block, wherein the conversion includes selectively enabling or disabling signaling of one or more syntax flags associated with the use of one or more of the following in response to detecting at least one condition: a joint chroma residual coding step, a luma-dependent chroma residual scaling (LDCRS) step, a secondary transform step, a quantization step, or an adaptive loop filter step, wherein a first chroma component of the current video block is associated with a first reduced secondary transform (RST) matrix, and a second chroma component of the current video block is associated with a second RST matrix.

[0032] In another representative aspect, the disclosed technology can be used to provide a method for video processing. The method includes performing a conversion between a current video block and a bitstream representation of the current video block, wherein the conversion includes: in response to detecting at least one condition, selectively enabling or disabling signaling of one or more syntax flags associated with the use of one or more of the following: a joint chroma residual coding step, a luma-dependent chroma residual residual (LDCRS) step, a secondary transform step, a quantization step, or an adaptive loop filter step, wherein a first chroma component of the current video block is associated with a first reduced secondary transform (RST) matrix, and a second chroma component of the current video block is associated with a second RST matrix; and in response to determining that the joint chroma residual coding step is enabled for the current video block, disabling signaling of a flag related to the use of CCLM on one or more chroma components of the current video block, wherein a flag in the one or more syntax flags is related to the use of a cross-component linear model (CCLM).

[0033] In yet another representative aspect, the disclosed technology may be used to provide a method of video processing, the method comprising: making a decision for a current video block regarding selectively including one or more codec block flags in a bitstream representation of the current video block; and performing a conversion between the current video block and the bitstream representation of the current video block based on the decision, wherein the selectivity includes being based on a color format, a component codec method of the current video block, or a codec mode of the current video block.

[0034] In yet another representative aspect, the disclosed technology can be used to provide a method of video processing, the method comprising: for a current video block, based on a plurality of non-zero coefficients in one or more color components of the current video block, making a decision about selectively applying a reduced secondary transform (RST) to the current video block; and performing a conversion between the current video block and a bitstream representation of the current video block based on the decision.

[0035] In yet another representative aspect, a video processing method is disclosed that includes performing conversion between a current video block and a bitstream representation of the current video block, wherein the bitstream representation selectively includes a syntax element indicating information about chroma codec based on a characteristic dependent on chroma.

[0036] In addition, in a representative aspect, a device in a video system is disclosed, the video system comprising a processor and a non-transitory memory having instructions thereon. The instructions executed by the processor cause the processor to implement any one or more of the disclosed methods.

[0037] Furthermore, a computer program product stored on a non-transitory computer-readable medium is disclosed, the computer program product comprising program code for performing any one or more of the disclosed methods.

[0038] The above and other aspects and features of the disclosed technology are described in more detail in the drawings, the description and the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 An example encoder block diagram for VVC is shown.

[0040] Figure 2 An example of a filter for a luminance component is shown.

[0041] Figure 3 An example flow chart of encoder decisions for a geometric transform based adaptive loop filter (GALF) is shown.

[0042] Figures 4A-4D An example of a subsampled Laplace calculation is shown.

[0043] Figure 5 An example decoder block diagram is shown.

[0044] Figure 6 An example of a secondary transform is shown.

[0045] Figure 7 An example of a reduced quadratic transform (RST) is shown.

[0046] Figure 8 An example of the nominal positions of luma and chroma samples in a picture of 4:2:0 format is shown.

[0047] Fig. 9 An example of the nominal positions of luma and chroma samples in a picture of 4:2:2 format is shown.

[0048] Fig.10 An example of the nominal positions of luma and chroma samples in a picture of 4:4:4 format is shown.

[0049] Fig.11 Examples of luma and chroma subsampling methods are shown.

[0050] Fig.12 Example relative positions of the top left chroma samples are shown.

[0051] Fig.13 Another example relative position of the top left chroma sample is shown.

[0052] Fig.14 Examples of sub-block transform (SBT) modes, SBT-V and SBT-H are shown.

[0053] Fig.15 An example of the SBT mode SBT-Q is shown.

[0054] Fig.16 is a block diagram of an example of a hardware platform for implementing the visual media decoding or visual media encoding techniques described in this document.

[0055] Fig.17 A flow chart of an example method for video encoding and decoding is shown.

[0056] Fig.18 is a diagram of the intra block copy mode.

[0057] Fig.19 An example of a block encoded and decoded in palette mode is shown.

[0058] Fig. 20 Examples of horizontal and vertical scanning are shown.

[0059] Fig.21 An example of encoding and decoding of palette indexes is shown.

[0060] Fig. 22 is a block diagram of an example video processing system in which the disclosed techniques may be implemented.

[0061] Fig.23 is a flowchart representation of a method for video decoding according to the present technology.

[0062] Fig.24 is a flowchart representation of a method for video encoding according to the present technology.

[0063] Fig.25 is a flowchart representation of another method for video decoding in accordance with the present technology.

[0064] Fig.26 is a flowchart representation of another method for video decoding in accordance with the present technology.

[0065] Fig. 27 is a flowchart representation of another method for video encoding in accordance with the present technology.

[0066] Fig.28 is a flowchart representation of another method for video encoding in accordance with the present technology.

[0067] Fig.29A is a flowchart representation of another method for video decoding in accordance with the present technology.

[0068] Fig.29B is a flowchart representation of another method for video encoding in accordance with the present technology.

[0069] Fig. 30A is a flowchart representation of another method for video decoding in accordance with the present technology.

[0070] Fig. 30B is a flowchart representation of another method for video encoding in accordance with the present technology.

[0071] Fig.31A is a flowchart representation of another method for video decoding in accordance with the present technology.

[0072] Fig.31B is a flowchart representation of another method for video encoding in accordance with the present technology.

[0073] Fig.32 is a flowchart representation of a method for video processing according to the present technology.

[0074] Fig.33 is a flowchart representation of another method for video processing according to the present technology.

[0075] Fig.34 is a flowchart representation of another method for video processing according to the present technology.

[0076] Fig.35 is a flowchart representation of another method for video processing according to the present technology.

[0077] Fig.36 is a flowchart representation of another method for video processing according to the present technology.

[0078] Fig.37 is a flowchart representation of another method for video processing according to the present technology.

[0079] Fig.38 is a flowchart representation of yet another method for video processing according to the present technology.

[0080] Fig.39 is a flowchart representation of yet another method for video processing according to the present technology.

[0081] Fig.40 is a flowchart representation of another method for video processing according to the present technology.

[0082] Fig.41 is a flowchart representation of another method for video processing according to the present technology.

[0083] Fig.42 is a flowchart representation of another method for video processing according to the present technology.

[0084] Fig.43 is a flowchart representation of yet another method for video processing according to the present technology. DETAILED DESCRIPTION

[0085] 1. Video Codec in HEVC / H.265

[0086] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T produced H.261 and H.263, ISO / IEC produced MPEG-1 and MPEG-4 Vision, and the two organizations jointly produced the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Coding (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure, in which temporal prediction and transform codecs are utilized. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and introduced them into a reference software named Joint Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG11 (MPEG) was created to work on the VVC standard, which targets a 50% bitrate reduction compared to HEVC.

[0087] 2.1 Color Space and Chroma Subsampling

[0088] A color space, also called a color model (or color system), is an abstract mathematical model that simply describes the color range as a tuple of numbers, usually 3 or 4 values ​​or color components (such as RGB). Basically, a color space is a refinement of coordinate systems and subspaces.

[0089] For video compression, the most commonly used color spaces are YCbCr and RGB.

[0090] YCbCr, Y′CbCr or Y Pb / Cb Pr / Cr, also written as YCbCr or Y′CbCr, is a family of color spaces used as part of the color picture pipeline in video and digital photography systems. Y′ is the luminance component, CB and CR are the blue difference and red difference chrominance components. Y′ (with superscript ') is different from Y luminance, which means that the light intensity is non-linearly encoded based on the gamma-corrected RGB primaries.

[0091] Chroma subsampling is the practice of encoding an image at a lower resolution for chroma information than for luminance information, taking advantage of the fact that the human visual system is less sensitive to color differences than to luminance. 2.1.1.4:4:4

[0093] Each of the three Y'CbCr components has the same sampling rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and in film post-production. 2.1.2.4:2:2

[0095] The two chroma components are sampled at half the sampling rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth for uncompressed video signaling by one third, with little visual difference 2.1.3.4:2:0

[0097] In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is halved because in this scheme, the Cb and Cr channels are sampled only on every alternating line. Therefore, the data rate is the same. Cb and Cr are each subsampled at 2x the speed in both the horizontal and vertical directions. There are three variations of the 4:2:0 scheme with different horizontal and vertical addressing.

[0098] In MPEG-2, Cb and Cr are co-addressed horizontally, and Cb and Cr are addressed between pixels in the vertical direction (intermittently addressed).

[0099] In JPEG / JFIF, H.261 and MPEG-1, Cb and Cr are addressed intermittently, intermediately between alternating luminance samples.

[0100] In 4:2:0 DV, Cb and Cr are co-located horizontally and vertically on alternate lines.

[0101] 2.2. Encoding and decoding process of typical video codecs

[0102] Figure 1 An example of an encoder block diagram for VVC is shown, which includes three loop filtering blocks: deblocking filter (DF), sample adaptive offset (SAO), and adaptive loop filter (ALF). Unlike DF, which uses a predefined filter, SAO and ALF utilize the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and applying a finite impulse response (FIR) filter, respectively, where codec-side information signals the offset and filter coefficients. ALF is located at the last processing stage for each picture and can be seen as a tool that attempts to capture and repair artifacts created by previous stages.

[0103] 2.3 Adaptive loop filter based on geometric transformation in JEM

[0104] In JEM, a geometric transform-based adaptive loop filter (GALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 2×2 block based on the direction and activity of the local gradient.

[0105] 2.3.1 Filter shape

[0106] In JEM, up to three diamond filter shapes can be selected for the luminance component (e.g. Figure 2 An index is signaled at the picture level to indicate the filter shape to use for the luma component.

[0107] For the chroma components in the picture, a 5x5 diamond shape is always used.

[0108] 2.3.1.1 Block Classification

[0109] Each 2×2 block is classified into one of 25 categories. The classification index C is based on its directionality D and activity The quantized value of is derived as follows:

[0110]

[0111] In order to calculate D and First, use the following one-dimensional Laplace formula to calculate the gradient in the horizontal, vertical and two diagonal directions:

[0112]

[0113]

[0114]

[0115]

[0116] The indices i and j refer to the coordinates of the top left sample in the 2×2 block, and R(i,j) represents the reconstructed sample at coordinates (i,j).

[0117] Then the maximum and minimum values ​​of the horizontal and vertical gradients of D are set to:

[0118]

[0119] And the maximum and minimum values ​​of the gradients in the two diagonal directions are set to:

[0120]

[0121] To derive the value of the directivity D, these values ​​are compared with each other and two thresholds t1 and t2 are used:

[0122] Step 1. If and are both true, then D is set to 0.

[0123] Step 2. If Then continue from step 3; otherwise continue from step 4.

[0124] Step 3. If Then D is set to 2; otherwise D is set to 1.

[0125] Step 4. If Then D is set to 4; otherwise D is set to 3.

[0126] The activity value A is calculated as follows:

[0127]

[0128] A is further quantized to the range of 0 to 4 (inclusive), and the quantized value is expressed as

[0129] For the two chrominance components in a picture, no classification method is applied, eg, a set of ALF coefficients is applied for each chrominance component.

[0130] 2.3.1.2. Geometric transformation of filter coefficients

[0131] Before filtering each 2×2 block, a geometric transformation (such as rotation or diagonal and vertical flipping) is applied to the filter coefficients f(k,l)f(k,l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks with ALF applied more similar by aligning their directionality.

[0132] Three geometric transformations are introduced, including diagonal, vertical flip and rotation:

[0133]

[0134] Where K is the size of the filter and 0≤k,l≤K-1 are the coefficient coordinates, so position (0,0) is in the upper left corner and position (K-1,K-1) is in the lower right corner. The transform is applied to the filter coefficients f(k,l) according to the gradient value calculated for the block. Table 1 summarizes the relationship between the transform and the four gradients in the four directions.

[0135] Table 1: Mapping of gradients computed for a block to transformations

[0136] Gradient Value Transform <![CDATA[g d2 <g d1 And g o <g ave ]]> No transformation <![CDATA[g d2 <g d1 And g ive <g h ]]> diagonal <![CDATA[g d1 <g d2 And g h <g v ]]> Flip Vertically <![CDATA[g d1 <g d2 And g v <g h ]]> Rotation

[0137] 2.3.1.3. Filter parameter signaling

[0138] In JEM, GALF filter parameters are signaled for the first CTU, for example, after the slice header and before the SAO parameters of the first CTU. Up to 25 sets of luminance filter coefficients can be signaled. In order to reduce bit overhead, filter coefficients of different classifications can be merged. In addition, the GALF coefficients of the reference picture are stored and allowed to be reused as GALF coefficients for the current picture. The current picture can choose to use the GALF coefficients stored for the reference picture and bypass the GALF coefficient signaling. In this case, only the index of one of the reference pictures is signaled, and the current picture inherits the stored GALF coefficients of the indicated reference picture.

[0139] To support GALF time domain prediction, a candidate list of GALF filter sets is maintained. At the beginning of decoding a new sequence, the candidate list is empty. After decoding a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (e.g., 6 in the current JEM), the new set of filters will overwrite the oldest set in the decoding order, that is, the first-in-first-out (FIFO) rule is applied to update the candidate list. To avoid duplication, a set can be added to the list only when the corresponding picture does not use GALF time domain prediction. In order to support time domain scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a time domain layer. More specifically, each array allocated by the time domain layer index (TempIdx) can constitute a filter set of a previously decoded picture with a lower TempIdx. For example, the kth array is allocated to be associated with a TempIdx equal to k, and it only includes filter sets from pictures with TempIdx less than or equal to k. After encoding or decoding a picture, the filter set associated with the picture will be used to update those arrays associated with equal or higher Templdx.

[0140] Temporal prediction of the GALF coefficients is used for inter-coded frames to minimize signaling overhead. For intra frames, temporal prediction is not available and a set of 16 fixed filters is assigned to each class. To indicate the use of fixed filters, a flag for each class is signaled and, if necessary, the index of the selected fixed filter. Even when a fixed filter is selected for a given class, the coefficients of the adaptive filter f(k,l) may still be sent for that class, in which case the coefficients of the filter that will be applied to the reconstructed picture are the sum of the two sets of coefficients.

[0141] The filtering process of the luma component can be controlled at the CU level. A flag is signaled to indicate whether GALF is applied to the luma component of the CU. For chroma components, whether GALF is applied is indicated only at the picture level.

[0142] 2.3.1.4. Filtering process

[0143] At the decoder side, when GALF is enabled for a block, each sample R(i,j) within the block is filtered to obtain the sample value R′(i,j) as shown below, where L represents the filter length and f m,n represents the filter coefficient, and f(k,l) represents the decoded filter coefficient.

[0144]

[0145] 2.3.1.5. Coding side filter parameter determination process

[0146] The overall encoder decision process of GALF is as follows: Figure 3 As shown. For each CU's luma samples, the encoder decides whether to apply GALF and includes the appropriate signaling flag in the slice header. For chroma samples, the decision to apply the filter is done on a picture level basis rather than on a CU level basis. In addition, the chroma GALF of a picture is only checked if luma GALF is enabled for the picture.

[0147] 2.4. Adaptive loop filter based on geometric transformation in VTM4.0

[0148] The current design of GALF in VVC has the following major changes compared to the current design of GALF in JEM:

[0149] (1) Adaptive filter shapes are removed. Only 7×7 filter shapes are allowed for luma components and 5×5 filter shapes are allowed for chroma components.

[0150] (2) Both the time domain prediction of the ALF parameters and the prediction from the fixed filter are removed.

[0151] (3) For each CTU, a one-bit flag signals whether ALF is enabled or disabled.

[0152] (4) The calculation of the class index is performed at the 4×4 level instead of the 2×2 level. In addition, the sub-sampled Laplacian calculation method for ALF classification can also be utilized. More specifically, there is no need to calculate the horizontal / vertical / 45 diagonal / 135 degree gradient for each sample within a block. Instead, a sub-sampling of 1:2 is used.

[0153] Figures 4A-4D Some examples of subsampled Laplacian calculations are shown.

[0154] 2.4.1 Filtering process

[0155] In VTM4.0, the filtering process of the adaptive loop filter is performed as follows:

[0156] O(x,y)=∑ (i,j) w(i,j)×I(x+i,y+j) (11)

[0157] Among them, sample I(x+i,y+j) is the input sample, O(x,y) is the output sample after filtering (for example, the filter result), and w(i,j) represents the filter coefficient. In fact, in VTM4.0, integer arithmetic is used to implement fixed-point precision calculation:

[0158]

[0159] Where L represents the filter length, and w(i,j) is the filter coefficient under fixed-point accuracy.

[0160] 2.5. Example Nonlinear ALF

[0161] 2.5.1 Filter Reshaping

[0162] Formula (11) can be reformulated as follows without affecting the encoding and decoding efficiency:

[0163] O(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j)×(I(x+i,y+j)-I(x,y)) (13)

[0164] where w(i,j) is the same as the filter coefficient in equation (11) [except that w(0,0) in equation (13) is equal to 1, while in equation (11) it is equal to 1-∑ (i,j)≠(0,0) w(i,j)].

[0165] 2.5.2. Modified filtering process

[0166] Using the filtering equation (13) above, when the difference between the neighboring sample value (I(x+i,y+j)) and the current sample value being filtered I(x,y) is too large, nonlinearity can be introduced to make the ALF more effective by using a simple limiting function to reduce the influence of the neighboring sample value.

[0167] In this proposal, the ALF filter is modified as follows:

[0168] O′(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j)×K(I(x+i,y+j)-I(x,y),k(i,j)) (14)

[0169] where K(d,b)=min(b,max(-b,d)) is the clipping function and k(i,j) is the clipping parameter, which depends on the filter coefficients associated with I(x+i,y+j). The encoder performs an optimization to find the best k(i,j).

[0170] In some embodiments, a clipping parameter k(i,j) is specified for each ALF filter, and one clipping value is signaled per filter coefficient. This means that up to 12 clipping values ​​can be signaled in the bitstream for each luma filter, and up to 6 clipping values ​​can be signaled for chroma filters.

[0171] To limit signaling costs and encoder complexity, the evaluation of the clipping value may be restricted to a small set of possible values.In some embodiments, 4 fixed values ​​may be used, which are the same for inter and intra slice groups.

[0172] Since the variance of local differences for luma is usually higher than for chroma, two different sets can be used for luma and chroma filters. The maximum sample value (here 1024, indicating a bit depth of 10 bits) can be included in each set so that clipping can be disabled if not necessary.

[0173] A set of clipping values ​​used in some embodiments is provided in Table 2. In the logarithmic domain, the 4 values ​​are selected by dividing the full range of sample values ​​for luma (coded as 10 bits) and the range of 4 to 1024 for chroma approximately equally.

[0174] More precisely, the brightness table of the limited radiation value is obtained by the following formula:

[0175] Where M = 2 10 And N=4.

[0176] Similarly, the chromaticity table of the limited radiation value is obtained according to the following formula:

[0177] Where M = 2 10 , N=4 and A=4.

[0178] Table 2: Authorized radiation limits

[0179]

[0180] The selected clipping value is encoded in the "alf_data" syntax element by using the Golomb coding scheme corresponding to the index of the clipping value in the above Table 2. This coding scheme is the same as that of the filter index.

[0181] 2.6. Exemplary CTB-based ALF

[0182] Three major changes are introduced in this proposal:

[0183] (1) Inherit filter coefficients from other ALF APSs (also called time domain prediction) in the CTB level.

[0184] (2) Predictive coding of filter coefficients from fixed filters.

[0185] (3) CTB level control is either using predictive codec from a fixed filter set or time domain prediction, or inherited from a signaled filter set (in the slice header).

[0186] 2.6.1. Strip-level temporal filter

[0187] Adaptive Parameter Set (APS) is adopted in VTM4. Each APS includes a set of signaled ALF filters, and up to 32 APSs are supported. In this proposal, time domain filters at the slice level are tested. The slice group can reuse the ALF information from the APS to reduce overhead. The APS is updated to a first-in-first-out (FIFO) buffer.

[0188] 2.6.2. Additional Exemplary CTB-Based ATF

[0189] For the luma component, when the ALF is applied to the luma CTB, it indicates whether to select from the prediction of 16 fixed filter sets or 5 temporal filter sets, or to inherit from a signaled filter set (in the slice header). Only the filter set index is signaled. For a slice, only one new set of 25 filters can be signaled. If a new set is signaled for a slice, all luma CTBs in the same slice share the set. The fixed filter set can be used to predict a new slice-level filter set and can also be used as a candidate filter set for the luma CTB. The total number of filters is 64.

[0190] For chroma components, when ALF is applied to chroma CTB, if a new filter is signaled for the slice, the CTB will use the new filter, otherwise the latest temporal chroma filter that meets the temporal scalability constraints will be applied.

[0191] As a slice-level time-domain filter, the APS is updated as a first-in-first-out (FIFO) buffer.

[0192] 2.6.3. Modified Specifications

[0193] The following text uses Fixed filter , Time Domain Filter and based on CTB The filter index of Modifications have been made.

[0194] 2.6.3.1. Syntax table

[0195] 7.3.3.2 Adaptive Loop Filter Data Syntax

[0196]

[0197]

[0198] 7.3.4.2. Codec Tree Unit Syntax

[0199]

[0200] 2.6.3.2. Semantics

[0201] 7.4.4.2. Adaptive Loop Filter Data Semantics

[0202] alf_signal_new_filter_luma alf_signal_new_filter_luma is equal to 0 and specifies not to signal a new luma filter set. alf_signal_new_filter_luma is 0 when not present.

[0203] alf_luma_use_fixed_filter_flag equal to 1 specifies that the fixed filter set is used for signaling the adaptive loop filter. alf_luma_use_fixed_filter_flag equal to 0 specifies that the fixed filter set is not used for signaling the adaptive loop filter.

[0204] alf_luma_fixed_filter_set_index Specifies the fixed filter set index. It can be 0…15.

[0205] alf_luma_fixed_filter_usage_pattern alf_luma_fixed_filter_usage_pattern equal to 1 specifies that some of the new filters use fixed filters and others do not.

[0206] alf_luma_fixed_filter_usage[i] alf_luma_fixed_filter_usage[i] is equal to 0 and specifies that the i-th filter does not use a fixed filter. alf_luma_fixed_filter_usage[i] is inferred to be 1 when not present.

[0207] alf_signal_new_filter_chromaalf_signal_new_filter_chroma is equal to 0 and specifies that a new chroma filter is not signaled.

[0208] alf_num_available_temporal_filter_sets_luma Specifies the number of available temporal filter sets available for the current slice, which can be 0 ... 5. It is 0 when not present.

[0209] The variable alf_num_available_filter_sets is derived as 16+alf_signal_new_filter_luma+alf_num_available_temporal_filter_sets_luma.

[0210] If alf_signal_new_filter_luma is 1, the following process occurs:

[0211] The variable filterCoefficients[sigFiltIdx][j] (where sigFiltIdx=0..alf_luma_num_filters_signaled_minus1, j=0..11) is initialized as follows:

[0212] filterCoefficients[sigFiltIdx][j]=alf_luma_coeff_delta_abs[sigFiltIdx][j]* (7-50)

[0213] (1-2*alf_luma_coeff_delta_sign[sigFiltIdx][j])

[0214] When alf_luma_coeff_delta_prediction_flag is equal to 1, filterCoefficients[sigFiltIdx][j] (where sigFiltIdx=1..alf_luma_num_filters_signaled_minus1 and j=0..11) is modified as follows:

[0215] filterCoefficients[sigFiltIdx][j]=alf_luma_coeff_delta_abs[sigFiltIdx][j]* (7-50)

[0216] (1-2*alf_luma_coeff_delta_sign[sigFiltIdx][j])

[0217] When alf_luma_coeff_delta_prediction_flag is equal to 1, filterCoefficients[sigFiltIdx][j] (where sigFiltIdx=1..alf_luma_num_filters_signaled_minus1 and j=0..11) is modified as follows:

[0218] filterCoefficients[sigFiltIdx][j]+=filterCoefficients[sigFiltIdx-1][j] (7-51)

[0219] Luminance filter coefficient AlfCoeff L With element AlfCoeff L [filtIdx][j] (where filtIdx=0..NumAlfFilters-1 and j=0..11) is derived as follows:

[0220] AlfCoeffL[filtIdx][j]=filterCoefficients[alf_luma_coeff_delta_idx[filtIdx]][j] (7-52)

[0221] If alf_luma_use_fixed_filter_flag is 1 and alf_luma_fixed_filter_usage [filtidx] is 1, then the following applies:

[0222] AlfCoeff L [filtIdx][j]=AlfCoeff L [filtIdx][j]+AlfFixedFilterCoeff [AlfClassToFilterMapping[alf_luma_fixed_filter_index][filtidx]][j]

[0223] For filtIdx = 0..NumAlfFilters–1, the final filter coefficient AlfCoeff L [filtIdx]

[12] is derived as follows:

[0224] AlfCoeffL[filtIdx]

[12] =128-Σk(AlfCoeffL[filtIdx][k]<<1), where k=0..11 (7-53)

[0225] Bitstream conformance requires that the value of AlfCoeffL[filtIdx][j] with filtIdx = 0..NumAlfFilters-1, j = 0..11 shall be between -2 7 To 2 7 -1 (inclusive), and the value of AlfCoeffL[filtIdx]

[12] should be between 0 and 2 8 The value is in the range of -1 (inclusive).

[0226] Luminance filter coefficients Alf Coeff LumaAll With element AlfCoeff LumaAll [filtSetIdx][filtIdx][j] (where filtSetIdx=0..15, filtSetIdx=0..NumAlfFilters-1 and j=0..12) is derived as follows:

[0227] Alf Coeff LumaAll [filtSetIdx][filtIdx][j]= AlfFixedFilterCoeff [AlfClassToFilterMapping[ filtSetIdx ][filtidx]][j]

[0228] Luminance filter coefficients Alf Coeff LumaAll With element AlfCoeff LumaAll [filtSetIdx][filtIdx][j] (where filtSetIdx=16, filtSetIdx=0..NumAlfFilters-1 and j=0..12) is derived as follows:

[0229] The variable closest_temporal_index is initialized to -1. Tid is the temporal layer index of the current stripe.

[0230] If alf_signal_new_filter_luma is 1

[0231] Alf Coeff LumaAll

[16] [filtIdx][j]=AlfCoeff L [filtIdx][j]

[0232] Otherwise, call the following procedure

[0233]

[0234] Alf Coeff LumaAll

[16] [filtIdx][j]=Temp L [closest_temporal_index][filtIdx][j]

[0235] Luminance filter coefficients Alf Coeff LumaAll With element AlfCoeff LumaAll[filtSetIdx][filtIdx][j] (where filtSetIdx=17..alf_num_available_filter_sets-1, filtSetIdx=0..NumAlfFilters-1 and j=0..12) is derived as follows:

[0236]

[0237]

[0238] - If alf_signal_new_filter_chroma is 1, the following process occurs:

[0239] The chroma filter coefficients AlfCoeffC[j] (where j=0..5) are derived as follows:

[0240] Alf Coeff C [j]=alf_chroma_coeff_abs[j]*(1-2*alf_chroma_coeff_sign[j])(7-57)

[0241] For j=6, the final filter coefficients are derived as follows:

[0242] Alf Coeff C [6] = 128-Σ k (Alf Coeff C [k]<<1), where k=0..5(7-58)

[0243] Bitstream conformance requirements have AlfCoeff j = 0..5 C The value of [j] should be between -2 7 To 2 7 -1 (including the end value), and AlfCoeff C [6] should be between 0 and 2 8 The value is in the range of -1 (inclusive).

[0244] - Otherwise (alf_signal_new_filter_chroma is 0) , call the following:

[0245]

[0246] Chroma filter coefficient AlfCoeff C [j] (where j = 0..6) is derived as follows:

[0247] Alf Coeff C [j]=Temp C [closest_temporal_index][j]

[0248] 7.4.5.2. Codec Unit Semantics

[0249] alf_luma_ctb_filter_set_index[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize]

[0250] The filter set index for the luma CTB at the specified position (xCtb, yCtb).

[0251] alf_use_new_filter When equal to 1, alf_luma_ctb_filter_set_index[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] is 16. When alf_use_new_filter is equal to 0, alf_luma_ctb_filter_set_index[xCtb>>Log2CtbSize][yCtb>>Log2CtbSize] is not 16.

[0252] alf_use_fixed_filter alf_use_fixed_filter is equal to 0 and specifies that the current luma CTB does not use any fixed filter set.

[0253] alf_fixed_filter_index Specifies the fixed filter set index, which can be from 0 to 15.

[0254] alf_temporal_index Specifies the temporal filter set index, which can be from 0 to alf_num_available_temporal_filter_sets_luma–1.

[0255] 2.6.3.3 Encoding and decoding process

[0256] 8.5.1 General

[0257] 1. When sps_alf_enabled_flag is equal to 0, the following applies:

[0258] - Invoke the time domain filter update process as specified in clause 8.5.4.5.

[0259] - To reconstruct the image sample array S L , S Cb and S Cr The reconstructed picture sample array S′ is the input and modified after sample adaptive offset L , S′ Cb and S′ Cr For output, the adaptive loop filter process specified in clause 8.5.4.1 is invoked.

[0260] - The array S'L , S′ Cb and S′ Cr Assigned to array S L , S Cb and S Cr (They represent decoded pictures).

[0261] - Invoke the time domain filter update process as specified in clause 8.5.4.6.

[0262] 8.5.4.2 Codec Tree Block Filtering Process for Luma Samples

[0263] - The array of luma filter coefficients f[j] corresponding to the filter specified by filtIdx[x][y] is derived as follows, where j=0..12:

[0264] f[j]= AlfCoeff LumaAll [alf_luma_ctb_filter_set_index[xCtb>>Log2CtbSize][yCtb>>

[0265] Log2CtbSize]]][filtIdx[x][y]][j] (8-732)

[0266] 8.5.4.5 Time Domain Filter Update

[0267] If any of the following conditions is true,

[0268] –The current image is an IDR image

[0269] – The current image is a BLA image

[0270] – In decoding order, the current picture is the first picture whose POC is greater than the POC of the last decoded IRAP picture, ie, after the previous picture and before the subsequent picture.

[0271] Then, temp_size_L and temp_size_C are set to 0.

[0272] 8.5.4.6 Time Domain Filter Update

[0273] If slice_alf_enabled_flag is 1 and alf_signal_new_filter_luma is 1, the following applies:

[0274] If the luma temporal filter buffer size temp_size_L<5, then temp_size_L=temp_size_L+1.

[0275] Temp L[i][j][k] (where i=temp_size_L−1…1, j=0…NumAlfFilters−1 and k=0…12) is updated as:

[0276] Temp L [i][j][k]=Temp L [i-1][j][k]

[0277] Temp L [i][j][k] (where j=0...NumAlfFilters-1 and k=0..12) is updated as:

[0278] Temp L [0][j][k] = AlfCoeff L [j][k]

[0279] Temp Tid_L [i] (where i = temp_size_L - 1 ... 1) is updated to:

[0280] Temp Tid_L [i]=Temp Tid_L [i-1]

[0281] Temp Tid_L [0] is set to the temporal layer index Tid of the current slice.

[0282] If alf_chroma_idx is not 0 and alf_signal_new_filter_chroma is 1, the following applies:

[0283] Temp c [i][j] (where i=temp_size_c−1…1 and j=0…6) is updated as:

[0284] Temp c [i][j]=Temp c [i-1][j]

[0285] Temp c [0][j] (where j=0...6) is updated to:

[0286] Temp c [0][j] = AlfCoeff C [j]

[0287] Temp Tid_C [i] (where i = temp_size_C – 1…1) is updated to:

[0288] TempTid_C [i]=Temp Tid_C [i-1]

[0289] Temp Tid_C [0] is set to the Tid of the current stripe.

[0290] Table 3: Syntax elements and associated binarization

[0291]

[0292]

[0293] Table 4: Assignment of ctxInc to syntax elements with context codec bins

[0294]

[0295] 2.7. Exemplary In-Loop Reshaping (ILR)

[0296] The basic idea of ​​inter-loop reshaping (ILR) is to convert the original (in the first domain) signaling (prediction / reconstruction signaling) to the second domain (the reshaped domain).

[0297] The loop luminance reshaper is implemented as a pair of lookup tables (LUTs), but only one of the two LUTs needs to be signaled, as the other can be calculated from the signaled LUT. Each LUT is a one-dimensional 10-bit 1024-entry mapping table (1D-LUT). One LUT is the forward LUT, FwdLUT, which converts the input luminance code value Y i Mapped to the changed value Y r : Y r =FwdLUT[Y i The other LUT is the inverse LUT, InvLUT, which changes the code value Y r Map to ( Represents Y i The reconstructed value of . ).

[0298] 2.7.1. Exemplary PWL Model

[0299] Conceptually, piecewise linear (PWL) is implemented as follows:

[0300] Let x1 and x2 be the two input pivot points, and y1 and y2 be their corresponding output pivot points. The output value y for any input value x between x1 and x2 can be interpolated by the following formula:

[0301] y=((y2-y1) / (x2-x1))*(x-x1)+y1

[0302] In fixed-point implementation, the formula can be rewritten as:

[0303] y=((m*x+2 FP_PREC-1 )>>FP_PREC)+c

[0304] Where m is a scalar, c is an offset, and FP_PREC is a constant value used to specify the precision.

[0305] Note that the PWL model can be used to pre-compute the 1024-entry FwdLUT and InvLUT mapping tables; however, the PWL model also allows the same mapping values ​​to be calculated on-the-fly without pre-computing the LUTs.

[0306] 2.7.2. Exemplary test model

[0307] 2.7.2.1 Brightness reshaping

[0308] In some embodiments, the test model of in-loop luma reshaping provides a lower complexity pipeline that also eliminates decoding delays for block-by-block intra prediction in inter slice reconstruction.Intra prediction is performed in the reshaped domain of both inter and intra slices.

[0309] Regardless of the slice type, intra prediction is always performed in the reshaped domain. With this arrangement, intra prediction can start immediately after the previous TU reconstruction is completed. This arrangement can also provide a unified process for intra modes instead of being slice-dependent. Figure 5 A block diagram showing an exemplary mode-based decoding process.

[0310] In some embodiments, luma and chroma residual scaling of a 16-segment piecewise linear (PWL) model may be tested instead of a 32-segment PWL model.

[0311] In some embodiments, inter-strip reconstruction with a loop luma reshaper may be implemented in test mode (light green shaded blocks indicate signaling in the reshaped domain: luma residual; predicted intra-luma; and reconstructed intra-luma).

[0312] 2.7.2.2. Luma-dependent Chroma Residual Scaling

[0313] Luma-dependent chroma residual scaling is a multiplication process implemented using fixed-point integer arithmetic. It is a scaling process in which a scaling factor is derived based on the reconstructed luma samples. The scaling factor is then applied to the chroma residual (i.e., chroma residual scaling). Chroma residual scaling compensates for the luma signaling that interacts with the chroma signaling. Chroma residual scaling is applied at the TU level. More specifically, the average value of the corresponding luma prediction block is utilized.

[0314] This average is used to identify an index in the PWL model. This index identifies the scaling factor cScaleInv. The chroma residuals are multiplied by this number.

[0315] Note that the chroma scaling factors are computed based on the predicted luma values ​​from the forward map instead of the reconstructed luma values.

[0316] 2.7.3. Use of ILR

[0317] At the encoder side, each picture (or slice group) is first converted to the reshaped domain. And all encoding and decoding processes are performed in the reshaped domain. For intra prediction, the neighboring blocks are in the reshaped domain; for inter prediction, the reference blocks (generated from the original domain from the decoded picture buffer) are first converted to the reshaped domain. Then the residual is generated and encoded and decoded to the bitstream.

[0318] After the entire picture (or slice group) has been encoded / decoded, the samples in the reshaped domain are converted to the original domain, and then the deblocking filter and other filters are applied.

[0319] Forward reshaping of prediction signaling is disabled for the following cases:

[0320] (1) The current block is intra-coded.

[0321] (2) The current block is encoded and decoded as CPR (Current Picture Reference, also known as Intra Block Copy, IBC).

[0322] (3) The current block is coded in combined inter-intra mode (CIIP) and forward reshaping of intra-predicted blocks is disabled.

[0323] 2.7.4. Example Signaling ILR Side Information

[0324] 2.7.4.1. Syntax table

[0325] LMCS APS: APS with aps_params_type equal to LMCS_APS

[0326] 7.3.2.3 Sequence parameter set RBSP syntax

[0327]

[0328]

[0329] 7.3.2.5 Adaptation parameter set syntax

[0330]

[0331] 7.3.5 Strip Header Syntax

[0332] 7.3.5.1 General Strip Header Syntax

[0333]

[0334]

[0335]

[0336] 7.3.5.4 Luma Mapping with Chroma Scaling Data Syntax

[0337]

[0338] 2.7.4.2. Semantics

[0339] sps_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is used in CVS. sps_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is not used in CVS.

[0340] Adaptive parameter set semantics

[0341] The adaptation_parameter_set_id provides an identifier of the APS for reference by other syntax elements. The APS can be shared across pictures and can be different in different slices within a picture.

[0342] aps_params_type specifies the type of APS parameters carried in the APS, as specified in Table 5.

[0343] Table 5: APS parameter type codes and APS parameter types

[0344]

[0345] slice_lmcs_enabled_flag equal to 1 specifies that luma mapping with chroma scaling is enabled for the current slice. slice_lmcs_enabled_flag equal to 0 specifies that luma mapping with chroma scaling is not enabled for the current slice. slice_lmcs_enabled_flag is inferred to be equal to 0 when not present.

[0346] slice_lmcs_aps_id specifies the adaptation_parameter_set_id of the LMCS APS to which the slice refers. The TemporalId of the LMCS APS NAL unit with adaptation_parameter_set_id equal to slice_lmcs_aps_id shall be less than or equal to the TemporalId of the slice NAL unit being coded or decoded.

[0347] When two or more slices of the same picture refer to multiple LMCS APSs having the same value of adaptation_parameter_set_id, the multiple LMCS APSs having the same value of adaptation_parameter_set_id should have the same content.

[0348] 7.4.6.4 Luma Mapping with Chroma Scaling Data Semantics

[0349] lmcs_min_bin_idx specifies the minimum bin index used in the luma map with chroma scaling build pass. The value of lmcs_min_bin_idx should be in the range of 0 to 15, inclusive.

[0350] lmcs_delta_max_bin_idx specifies a delta value between 15 and the maximum bin index LmcsMaxBinIdx used in luma mapping with chroma scaling build process. The value of lmcs_delta_max_bin_idx shall be in the range of 0 to 15, inclusive. The value of LmcsMaxBinIdx is set equal to 15 - lmcs_delta_max_bin_idx. The value of LmcsMaxBinIdx shall be greater than or equal to lmcs_min_bin_idx.

[0351] lmcs_delta_cw_prec_minus1 plus 1 specifies the number of bits used to represent the syntax lmcs_delta_abs_cw[i]. The value of lmcs_delta_cw_prec_minus1 shall be in the range of 0 to BitDepthY-2, inclusive.

[0352] lmcs_delta_abs_cw[i] specifies the absolute delta codeword value for the i-th bin.

[0353] lmcs_delta_sign_cw_flag[i] specifies the sign of the variable lmcsDeltaCW[i] as follows:

[0354] - If lmcs_delta_sign_cw_flag[i] is equal to 0, lmcsDeltaCW[i] is a positive value.

[0355] - Otherwise (lmcs_delta_sign_cw_flag[i] is not equal to 0), lmcsDeltaCW[i] is a negative value.

[0356] lmcs_delta_sign_cw_flag[i] is inferred to be equal to 0 when not present.

[0357] The variable OrgCW is derived as follows:

[0358] OrgCW=(1< <BitDepth Y ) / 16 (7-77)

[0359] The variable lmcsDeltaCW[i] (where i=lmcs_min_bin_idx..LmcsMaxBinIdx) is derived as follows:

[0360] lmcsDeltaCW[i]=(1-2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i](7-78)

[0361] The variable lmcsCW[i] is derived as follows:

[0362] - For i = 0..lmcs_min_bin_idx–1, lmcsCW[i] is set equal to 0.

[0363] - For i = lmcs_min_bin_idx..LmcsMaxBinIdx, the following applies:

[0364] lmcsCW[i]=OrgCW+lmcsDeltaCW[i] (7-79)

[0365] The value of lmcsCW[i] should be in the range of (OrgCW>>3) to (OrgCW<<3-1) (inclusive).

[0366] - For i=LmcsMaxBinIdx+1..15, lmcsCW[i] is set equal to 0.

[0367] Meeting the following conditions is a requirement for bitstream conformance:

[0368]

[0369] The variable InputPivot[i] (where i=0..16) is derived as follows:

[0370] InputPivot[i]=i*OrgCW

[0371] The variables LmcsPivot[i] (where i=0..16), ScaleCoeff[i] and InvScaleCoeff[i] (where i=0..15) are derived as follows:

[0372]

[0373] The variable ChromaScaleCoeff[i] (where i=0...15) is derived as follows:

[0374] chromaResidualScaleLut[]={16384,16384,16384,16384,16384,16384,16384,8192,8192,8192,8192, 5461,5461,5461,5461,4096,4096,4096,4096,3277,3277,3277,3277,2731,2731,2731,2731,2341,234 1,2341,2048,2048,2048,1820,1820,1820,1638,1638,1638,1489,1489,1489,1365,1365,1365,1260,1260,1260,1170,1170,1170,1092,1092,1092,1024,1024,1024,1024}

[0375]

[0376] The variables ClipRange, LmcsMinVal, and LmcsMaxVal are derived as follows:

[0377] ClipRange=((lmcs_min_bin_idx>0)&&(LmcsMaxBinIdx<15) (7-84)

[0378] LmcsMinVal=16<<(BitDepth Y -8) (7-85)

[0379] LmcsMaxVal=235<<(BitDepth Y-8) (7-86)

[0380] NOTE - the arrays InputPivot[i] and LmcsPivot[i], ScaleCoeff[i], as well as InvScaleCoeff[i], ChromaScaleCoeff[i], ClipRange, LmcsMinVal and LmcsMaxVal are updated only if slice_lmcs_model_present_flag is equal to 1. So, for example, the lmcs model can be sent with an IRAP picture, but lmcs is disabled for that IRAP picture.

[0381] 2.8. Example Joint Coding and Decoding of Chroma Residual

[0382] This test measures the performance of an exemplary joint Cb-Cr codec mode where only the joint residual is encoded.

[0383] If the chroma reshaper is active, the reshaping is applied to the received residual, the same as done in separate codec mode (i.e., reshaping joint residual signaling). On the encoder side, when testing this mode, the average of the positive Cb residual and the negative Cr residual is used as the joint residual:

[0384] resJoint=(resCb–resCr) / 2

[0385] A bin indicator is signaled in the bitstream to enable this mode. In case this mode is enabled, the joint residual signaling is encoded in the bitstream. At the decoder side, the joint residual is used for the Cb component, while the negative version of the residual is applied to Cr.

[0386] Signaling of this mode is expected to follow the syntax and semantics given below:

[0387]

[0388] tu_cb_cr_joint_residual[x0][y0] specifies whether the indicated Cb residual is used to derive both Cb and Cr residuals. The array indices x0, y0 specify the position (x0, y0) of the top left luma sample of the considered transform block relative to the top left luma sample of the picture.

[0389] 7.3.2.2 Picture parameter set RBSP syntax

[0390]

[0391] 7.3.4.1 General slice group header syntax

[0392]

[0393] 7.3.6.12 Residual Codec Syntax

[0394]

[0395]

[0396] 2.9 Reduced quadratic transformation

[0397] In JEM, a secondary transform is applied between the forward primary transform and quantization (at the encoder) and between dequantization and the inverse primary transform (at the decoder side). Figure 6 As shown, a 4×4 (or 8×8) secondary transform is performed, depending on the block size. For example, for each 8×8 block, a 4×4 secondary transform is applied to smaller blocks (e.g., min(width, height) < 8) and an 8×8 secondary transform is applied to larger blocks (e.g., min(width, height) > 4).

[0398] For the secondary transform, a non-separable transform is applied, hence also referred to as a non-separable secondary transform (NSST). There are 35 transform sets in total, and each transform set uses 3 non-separable transform matrices (kernels, each kernel having a 16×16 matrix).

[0399] In some embodiments, a reduced secondary transform (RST) and 4 transform sets (instead of 35 transform sets) mapping according to the intra prediction direction can be used. In some cases, 16×48 and 16×16 matrices are used for 8×8 and 4×4 blocks, respectively. For ease of annotation, the 16×48 transform is represented as RST8×8, and the 16×16 transform is represented as RST 4×4. This approach has recently been adopted by VVC.

[0400] Figure 7 An example of a reduced secondary transform (RST) is shown. The secondary forward and inverse transforms are separate process steps from the primary transform.

[0401] For the encoder, a forward transform is first performed, followed by a second forward transform and quantization, and CABAC bit encoding. For the decoder, CABAC bit decoding and inverse quantization are first performed, followed by a second inverse transform, and then a first inverse transform.

[0402] RST is only applicable to intra-frame codec Tus.

[0403] 2.9.1. Syntax and semantics

[0404] 7.3.2.1 Sequence Parameter Set RBSP Syntax

[0405]

[0406]

[0407] 7.3.6.11 Residual Codec Syntax

[0408]

[0409]

[0410]

[0411]

[0412] 7.3.6.5 Codec unit syntax

[0413]

[0414]

[0415]

[0416] 7.4.3.1 Sequence Parameter Set RBSP Semantics

[0417] sps_st_enabled_flag equal to 1 specifies that st_idx may be present in the residual codec syntax of an intra codec unit. sps_st_enabled_flag equal to 0 specifies that st_idx is not present in the residual codec syntax of an intra codec unit.

[0418] 7.4.7.5 Codec unit semantics

[0419] st_idx[x0][y0] specifies which secondary transform kernel is applied between the two candidate kernels in the selected transform set. st_idx[x0][y0] equal to 0 specifies that no secondary transform is applied. Array indices x0, y0 specify the position (x0, y0) of the top left sample of the transform block under consideration relative to the top left sample of the picture.

[0420] When st_idx[x0][y0] does not exist, st_idx[x0][y0] is inferred to be equal to 0.

[0421] 2.10. Support of various color formats in VVC

[0422] 2.10.1. Syntax, semantics, and decoding process

[0423] 7.3.2.3 Sequence parameter set RBSP syntax

[0424]

[0425]

[0426]

[0427] chroma_format_idc specifies the chroma samples relative to the luma samples specified in clause 6.2. The value of chroma_format_idc shall be in the range 0 to 3, inclusive.

[0428] separate_colour_plane_flag equal to 1 specifies that the three colour components of the 4:4:4 chroma format are coded separately. separate_colour_plane_flag equal to 0 specifies that the colour components are not coded separately. When separate_colour_plane_flag is not present, it is inferred to be equal to 0. When separate_colour_plane_flag is equal to 1, the codec picture consists of three separate components, each consisting of coded samples of one colour plane (Y, Cb or Cr), and uses the monochrome codec syntax. In this case, each colour plane is associated with a specific colour_plane_id value.

[0429] There is no dependency in the decoding process between colour planes with different colour_plane_id values. For example, the decoding process of a monochrome picture with one colour_plane_id value does not use any data from a monochrome picture with a different colour_plane_id value for inter prediction.

[0430] Depending on the value of separate_colour_plane_flag, the value of the variable ChromaArrayType is assigned as follows:

[0431] - If separate_colour_plane_flag is equal to 0, ChromaArrayType is set equal to chroma_format_idc.

[0432] - Otherwise (separate_colour_plane_flag is equal to 1), ChromaArrayType is set equal to 0.

[0433] 6.2 Source, Decoding and Output Image Formats

[0434] This section specifies the relationship between source pictures given by the bitstream and the decoded pictures.

[0435] The video source represented by a bitstream is a sequence of pictures arranged in decoding order.

[0436] Both source and decoded pictures consist of one or more sample arrays:

[0437] - Luminance (Y) only (monochrome).

[0438] - Luma and two chrominances (YCbCr or YCgCo).

[0439] - Green, Blue and Red (GBR, also known as RGB).

[0440] - Arrays representing other unspecified monochromatic or tristimulus color sampling (e.g., YZX, also called XYZ).

[0441] For the convenience of notation and terminology in this specification, the variables and terms associated with these arrays are referred to as luminance (either L or Y) and chrominance, with the two chrominance arrays being referred to as Cb and Cr; regardless of the actual color representation method used. The actual color representation method used can be indicated using the syntax specified in Appendix TBD.

[0442] Depending on the chroma format sampling structure (specified by chroma_format_idc and separate_colour_plane_flag), the variables SubWidthC and SubHeightC are specified in Table 6. In the future, ITU-T|ISO / IEC may specify other values ​​for chroma_format_idc, SubWidthC and SubHeightC.

[0443] Table 6 — SubWidthC and SubHeightC values ​​derived from chroma_format_idc and separate_colour_plane_flag

[0444]

[0445] In monochrome sampling, there is only one sample array, nominally considered to be the brightness array.

[0446] In 4:2:0 sampling, each of the two chroma arrays has half the height and half the width of the luma array.

[0447] In 4:2:2 sampling, each of the two chroma arrays has the same height and half the width of the luma array.

[0448] In 4:4:4 sampling, depending on the value of separate_colour_plane_flag, the following applies:

[0449] - If separate_colour_plane_flag is equal to 0, each of the two chroma arrays has the same height and width as the luma array.

[0450] - Otherwise (separate_colour_plane_flag is equal to 1), the three colour planes are processed as monochrome sampled pictures.

[0451] The number of bits required to represent each sample in the luma and chroma arrays of a video sequence is between 8 and 16 (inclusive), and the number of bits used in the luma array may be different from the number of bits used in the chroma arrays.

[0452] When the value of chroma_format_idc is equal to 1, Figure 8 The nominal vertical and horizontal relative positions of luma and chroma samples in the picture are shown. Alternative chroma sample relative positions may be indicated in the video usability information.

[0453] When the value of chroma_format_idc is equal to 2, the chroma samples are co-located with the corresponding luma samples, and the nominal positions in the picture are as follows: Fig. 9 shown.

[0454] When the value of chroma_format_idc is equal to 3, all array samples of a picture are co-located in all cases, and the nominal positions in the picture are as follows Fig.10 shown.

[0455] Appendix E: Video Availability Information

[0456] Even for the same chroma subsampling format, such as 4:2:0, the subsampling method can have multiple different ways. Fig.11 Mark the different types.

[0457] Fig.12 The diagram shows the indicated relative position of the top left chroma sample when chroma_format_idc is equal to 1 (4:2:0 chroma format) and chroma_sample_loc_type is equal to the value of the variable ChromaLocType. The area represented by the top left 4:2:0 chroma sample (depicted as a large red square with a large red dot at its center) is shown relative to the area represented by the top left luma sample (depicted as a small black square with a small black dot at its center). The areas represented by the neighboring luma samples are depicted as small gray squares with small gray dots at their centers.

[0458] like Fig.13As shown, the relative spatial positioning of the chroma samples can be represented by defining two variables HorizontalOffsetC and VerticalOffsetC as functions of chroma_format_idc and the variable ChromaLocType as given in Table 7, where HorizontalOffsetC is the horizontal (x) position of the center of the upper left chroma sample relative to the center of the upper left luma sample in units of luma samples; VerticalOffsetC is the vertical (y) position of the center of the upper left chroma sample relative to the center of the upper left luma sample in units of luma samples.

[0459] In a typical FIR filter design, when chroma_format_idc is equal to 1 (4:2:0 chroma format) or 2 (4:2:2 chroma format), HorizontalOffsetC and VerticalOffsetC will be used as the phase offsets for the horizontal and vertical filter operations, respectively, for separable downsampling from the 4:4:4 chroma format to the chroma format indicated by chroma_format_idc.

[0460] Table 7 – Definition of HorizontalOffsetC and VerticalOffsetC as a function of chroma_format_idc and ChromaLocType

[0461]

[0462] When chroma_format_idc is equal to 1 (4:2:0 chroma format) and the decoded video content is intended to be interpreted according to Rec. ITU-R BT.2020-2 or Rec. ITU-R BT.2100-1, chroma_loc_info_present_flag shall be equal to 1 and chroma_sample_loc_type shall be equal to 2.

[0463] 2.11. Example Extended ALF

[0464] In some embodiments, the nonlinear ALF concept can be extended. More precisely, some syntax elements can be added to enable the use of alternative ALF luma filter sets and alternative chroma filters and select them on a per-filter, per-CTU basis. For each CTU, if ALF filtering is enabled on the luma component, an additional luma filter set alternative index is signaled for each ALF luma filter index. Similarly, for each CTU, for each chroma component for which ALF filtering is enabled, an alternative chroma filter index is signaled to select a given filter for the chroma component.

[0465] 2.12 Exemplary Sub-Block Transform (SBT) in VVC

[0466] For an inter-predicted CU with cu_cbf (also called cu_coded_flag) equal to 1, cu_sbt_flag can be signaled to indicate whether to decode the entire residual block or a sub-part of the residual block. In the former case, the information of the inter MTS is further parsed to determine the transform type of the CU. In the latter case, a portion of the residual block is encoded and decoded with the inferred adaptive transform, and the other parts of the residual block are zeroed. SBT should not be used to combine inter and intra modes because almost no coding gain is achieved.

[0467] 2.12.1 Sub-block TU tiling

[0468] When SBT is used for inter CU, SBT type and SBT position information are further decoded from the bitstream. There are two SBT types and two SBT positions, such as Fig.14 As shown. For SBT-V (or SBT-H), the TU width (or height) can be equal to half of the CU width (or height) or 1 / 4 of the CU width (or height), signaled by another flag, resulting in 2:2 partitioning or 1:3 / 3:1 partitioning. The 2:2 partitioning is similar to the binary tree (BT) partitioning, while the 1:3 / 3:1 partitioning is similar to the asymmetric binary tree (ABT) partitioning. If one side of the CU is 8 in the luma sample, 1:3 / 3:1 partitioning along that side is not allowed. Therefore, a CU has a maximum of 8 SBT modes.

[0469] Quadtree (QT) partitioning is further used to tile a CU into 4 sub-blocks, and there is still one sub-block with residual, such as Fig.15 This SBT type is denoted as SBT-Q. This part is not used in VVC.

[0470] SBT-V, SBT-H, and SBT-Q are allowed for CUs whose width and height are not greater than maxSbtSize. maxSbtSize is signaled in the SPS. For HD and 4K sequences, the encoder sets maxSbtSize to 64; for other sequences with smaller resolutions, maxSbtSize is set to 32.

[0471] 2.12.2 Sub-block Transform Type

[0472] Position-dependent transforms are applied to luma transform blocks in SBT-V and SBT-H (chroma TBs always use DCT-2). The two positions of SBT-H and SBT-V are associated with different core transforms. More specifically, Fig.14The horizontal and vertical transforms for each SBT position are specified in . For example, the horizontal and vertical transforms for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is larger than 32, the corresponding transform is set to DCT-2. Therefore, the sub-block transform jointly specifies the TU tiling, cbf / codec flags, and horizontal and vertical transforms for the residual block, which can be seen as a syntax shortcut for the case where the main residual of the block is on one side of the block.

[0473] 2.12.3 Sub-block Transformation in VVC

[0474] For an inter-predicted CU with cu_cbf (or cu_coded_flag) equal to 1 (e.g., with at least one non-zero coefficient), cu_sbt_flag can be signaled to indicate whether to decode the entire residual block or a sub-part of the residual block. In the former case, the inter MTS information is further parsed to determine the transform type of the CU. In the latter case, part of the residual block is encoded and decoded with the inferred adaptive transform, and the other parts of the residual block are zeroed. SBT should not be used for combined inter-intra modes because almost no coding gain is achieved.

[0475] When SBT is used for inter CU, SBT type and SBT position information are further decoded from the bitstream. For SBT-V (or SBT-H), the TU width (or height) can be equal to half of the CU width (or height) or 1 / 4 of the CU width (or height), signaled by another flag, resulting in 2:2 partitioning or 1:3 / 3:1 partitioning. The 2:2 partitioning is similar to the binary tree (BT) partitioning, while the 1:3 / 3:1 partitioning is similar to the asymmetric binary tree (ABT) partitioning. If one side of the CU is 8 in the luminance sample, 1:3 / 3:1 partitioning along that side is not allowed. Therefore, a CU has a maximum of 8 SBT modes.

[0476] Position-dependent transforms are applied to luma transform blocks in SBT-V and SBT-H (chroma TBs always use DCT-2). The two positions of SBT-H and SBT-V are associated with different core transforms. More specifically, Fig.14 The horizontal and vertical transforms for each SBT position are specified in . For example, the horizontal and vertical transforms for SBT-V position 0 are DCT-8 and DST-7, respectively. When one side of the residual TU is greater than 32, the corresponding transform is set to DCT-2. Therefore, the sub-block transform jointly specifies the TU tiling, cbf / codec flags, and horizontal and vertical transforms for the residual block, which can be seen as a syntax shortcut for the case where the main residual of the block is on one side of the block.

[0477] 7.3.4.6 Codec unit syntax

[0478]

[0479]

[0480] 7.3.4.10 Transformation Tree Syntax

[0481]

[0482]

[0483] 7.3.4.11 Transformation unit syntax

[0484]

[0485]

[0486]

[0487] 2.13 Intra-frame Block Copy

[0488] Intra-block copy (IBC), also known as current picture reference, has been adopted in HEVC Screen Content Codec Extension (HEVC-SCC) and the current VVC Test Model (VTM-4.0). IBC extends the concept of motion compensation from inter-frame coding to intra-frame coding. Fig.18 As shown ( Fig.18 is an illustration of intra-frame block copying), when IBC is applied, the current block is predicted by a reference block in the same picture. The samples in the reference block must have been reconstructed before the current block is encoded or decoded. Although IBC is not efficient for sequences captured by most cameras, it shows significant codec gains for screen content. The reason is that there are many repeated patterns in screen content pictures, such as icons and text characters. IBC can effectively eliminate the redundancy between these repeated patterns. In HEVC-SCC, an inter-frame codec unit (CU) can apply IBC if it selects the current picture as its reference picture. In this case, MV is renamed block vector (BV), and BV always has integer pixel precision. For compatibility with the main profile HEVC, the current picture is marked as a "long-term" reference picture in the decoded picture buffer (DPB). It should be noted that similarly, in the multi-view / 3D video codec standard, inter-view reference pictures are also marked as "long-term" reference pictures.

[0489] Once the BV has found its reference block, a prediction can be generated by copying the reference block. The reference pixels can be subtracted from the original signaling to get the residual. Transforms and quantization can then be applied as in other codec modes.

[0490] However, when the reference block is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area subject to some constraints, some or all pixel values ​​are not defined. Basically, there are two solutions to such a problem. One is to prohibit this situation, for example in terms of bitstream consistency. The other is to apply padding to those undefined pixel values. The following subsections describe the solutions in detail.

[0491] 2.13.1 IBC in the VVC Test Model

[0492] In the current VVC test model, such as the VTM-4.0 design, the entire reference block should be the current codec tree unit (CTU) and does not overlap with the current block. Therefore, there is no need to fill the reference block or prediction block. The IBC flag is encoded and decoded as the prediction mode of the current CU. Therefore, there are three prediction modes for each CU, namely MODE_INTRA, MODE_INTER and MODE_IBC.

[0493] 2.13.1.1 IBC Merge Mode

[0494] In IBC merge mode, the index pointing to the entry in the IBC merge candidate list is parsed from the bitstream. The construction of the IBCmerge list can be summarized as the sequence of the following steps:

[0495] Step 1: Derive Spatial Candidates

[0496] Step 2: Insert HMVP candidates

[0497] Step 3: Insert pairwise average candidates

[0498] In the derivation of spatial merge candidates, such as Figure 2 As depicted, up to four merge candidates are selected from the candidates at the positions depicted by A1, B1, B0, A0, and B2. The order of derivation is A1, B1, B0, A0, and B2. Position B2 is considered only when any PU at position A1, B1, B0, A0 is not available (for example, because it belongs to another slice or piece) or IBC mode encoding is not used. After adding the candidate at position A1, the insertion of the remaining candidates is checked for redundancy, which ensures that candidates with the same motion information are excluded from the list, thereby improving encoding and decoding efficiency.

[0499] After inserting the spatial candidates, if the IBC merge list size is still less than the maximum IBC merge list size, the IBC candidates in the HMVP table can be inserted. Redundancy checks are performed when inserting HMVP candidates.

[0500] Finally, the pairwise averaged candidates are inserted into the IBC merge list.

[0501] When the reference block identified by the merge candidate is outside the picture, or overlaps with the current block, or is outside the reconstructed area, or is outside the valid area subject to certain constraints, the merge candidate is called an invalid merge candidate.

[0502] Note that invalid merge candidates may be inserted into the IBC merge list.

[0503] 2.13.1.2 IBC AMVP Model

[0504] In IBC AMVP mode, the AMVP index points pointing to the entries in the IBC AMVP list are parsed from the bitstream. The construction of the IBC AMVP list can be summarized in the sequence of the following steps:

[0505] Step 1: Derivation of Spatial Candidates

[0506] - Check A0, A1 until a usable candidate is found.

[0507] - Check B0, B1, B2 until a usable candidate is found.

[0508] Step 2: Insert HMVP candidates

[0509] Step 3: Insert zero candidates.

[0510] After inserting the spatial candidates, if the IBC AMVP list size is still smaller than the maximum IBC AMVP list size, the IBC candidates in the HMVP table can be inserted.

[0511] Finally, zero candidates are inserted into the IBC AMVP list.

[0512] 2.13.1.3 Chroma IBC Mode

[0513] In the current VVC, motion compensation in the chroma IBC mode is performed at the sub-block level. The chroma block will be divided into several sub-blocks. Each sub-block determines whether the corresponding luminance block has a block vector and the validity of the block vector when it exists. There is an encoder constraint in the current VTM, where the chroma IBC mode will be tested if all sub-blocks in the current chroma CU have a valid luminance block vector. For example, on YUV420 video, the chroma block is N×M, and then the collocated luminance area is 2N×2M. The sub-block size of the chroma block is 2×2. There are several steps to perform the chroma mv derivation and then the block copy process.

[0514] 1) First, split the chrominance block into (N>>1)*(M>>1) sub-blocks.

[0515] 2) Each sub-block with an upper left sample at coordinates (x, y) gets a corresponding luma block that covers the same upper left sample at coordinates (2x, 2y).

[0516] 3) The encoder checks the block vector (bv) of the acquired luminance block. If one of the following conditions is met, bv is considered invalid.

[0517] a. The bv of the corresponding luminance block does not exist.

[0518] b. The prediction block identified by bv has not yet been reconstructed.

[0519] c. The prediction block identified by bv partially or completely overlaps with the current block.

[0520] 4) Set the chrominance motion vector of the sub-block to the motion vector of the corresponding luminance sub-block.

[0521] When valid bv is found for all sub-blocks, the encoder allows IBC mode.

[0522] 2.14 Palette Mode in HEVC Screen Content Codec Extension (HEVC-SCC)

[0523] The basic idea behind the palette mode is that samples in a CU are represented by a small set of representative color values. This set is called the palette. Samples outside the palette can also be indicated by signaling an escape symbol and subsequent (possibly quantized) component values. This is illustrated in Fig.19 middle.

[0524] In the palette mode of HEVC-SCC, a prediction method is used to encode and decode the palette and index mapping.

[0525] 2.14.1 Encoding and decoding of palette index

[0526] The palette index is encoded and decoded using horizontal and vertical traversal scans, such as Fig. 20 The scanning order is explicitly signaled in the bitstream using palette_transpose_flag. For the rest of this subclause, it is assumed that the scanning is horizontal.

[0527] The palette index is encoded and decoded using two main palette sample modes: "INDEX" and "COPY_ABOVE". As mentioned before, escape symbols are also signaled as "INDEX" mode and are assigned an index equal to the maximum palette size. Except for the top row or when the previous mode was "COPY_ABOVE", a flag is used to signal the mode. In "COPY_ABOVE" mode, the palette index of the sample in the row above is copied. In "INDEX" mode, the palette index is signaled explicitly. For both "INDEX" and "COPY_ABOVE" modes, a run value is signaled that specifies the number of subsequent samples that are also encoded and decoded using the same mode. When the escape symbol is part of a run in "INDEX" or "COPY_ABOVE" mode, the escape component value is signaled for each escape symbol. The encoding and decoding of the palette index is as follows: Fig.21 shown.

[0528] The syntax sequence is completed as follows. First, the number of index values ​​for the CU is signaled. This is followed by signaling the actual index value for the entire CU using truncated binary encoding. Both the index number and the index value are encoded and decoded in bypass mode. This groups together the bypass bins associated with the index. The palette sampling mode (if necessary) and operation are then signaled in an interleaved manner. Finally, the component escape values ​​corresponding to the escape samples for the entire CU are grouped together and encoded and decoded in bypass mode.

[0529] After signaling the index value, an additional syntax element last_run_type_flag is signaled. This syntax element, combined with the index number, eliminates the need to signal the run value corresponding to the last run in the block.

[0530] In HEVC-SCC, palette mode is also enabled for 4:2:2, 4:2:0 and monochrome chroma formats. The signaling of palette entries and palette indices is almost the same for all chroma formats. In the case of non-monochrome formats, each palette entry consists of 3 components. For monochrome formats, each palette entry consists of a single component. For the subsampled chroma direction, chroma samples are associated with luma sample indices that are divisible by 2. After reconstructing the palette index of the CU, if the sample has only one component associated with it, only the first component of the palette entry is used. The only difference in the signaling is the escape component values. For each escape sample, the number of escape component values ​​signaled may be different, depending on the number of components associated with the sample.

[0531] 2.15 Double Tree Palette Mode

[0532] When dual tree is enabled, whether to use palette mode for chroma blocks is signaled separately from the signaling for luma blocks.

[0533] 2. Disadvantages of existing implementations

[0534] The current VVC design has the following problems:

[0535] (1) The joint coding method of chroma residual has a problem in that signaling of enabling / disabling the method and signaling of a chroma delta quantization parameter (QP) used in the method are applied even when the chroma component is not available.

[0536] (2) When dual-tree is enabled, the current design of IBC and palette mode requires signaling of the IBC / palette mode of the chroma codec block even if the sequence is 4:0:0.

[0537] (3) The method of RST has a problem in that even in the case where the chrominance component is not available, signaling of enabling / disabling the method and signaling of the transform matrix index are applied.

[0538] (4) Regardless of the value of ChromaArrayType, slice_chroma_remain_scale_flag and the number of bits used to represent each PCM sample value of the chroma component (e.g., PCM_sample_bit_depth_chroma_minus1) are signaled.

[0539] (5) For each CTU, for each chroma component on which ALF filtering is enabled, signal an alternative chroma filter index to select a given filter for the chroma component. This signaling method does not check the availability of the chroma component.

[0540] (6) Regardless of the value of ChromaArrayType, the signaling notification of pps_cb_qp_offset and pps_cr_qp_offset in the picture parameter set (PPS) will be completed.

[0541] (7) Both color components always share the same RST matrix. At the same time, RST and joint chroma residual coding may be applied together, which is suboptimal.

[0542] (8) The non-zero coefficient count threshold in RST follows the following rules:

[0543] a. If it is a separate tree codec, for the luma component, when there is at least one non-zero coefficient, the RST index of the luma component may be signaled;

[0544] b. If it is a separate tree codec, for two chroma components, once there is at least one non-zero coefficient in the two chroma blocks (e.g. Cb and Cr), it is possible to signal the RST index of the two chroma components;

[0545] c. In case of single-tree coding, once there are at least 2 non-zero coefficients in three blocks (e.g. Y, Cb and Cr), it is possible to signal the RST index of the entire coding unit (including three color components);

[0546] May need to be consistent with color format and whether separate plane codecs are enabled.

[0547] (9) The conditional signaling notification of the SBT flag should be consistent with the color format and whether separate plane codec is enabled.

[0548] (10) For inter-frame coded blocks, it may happen that cu_cbf (or cu_coded_flag) is equal to 1, but the coded block flags of all three components (tu_cbf_luma / tu_y_coded_flag, tu_cbf_cb / tu_cb_coded_flag, tu_cbf_cr / tu_cr_coded / flag) are all zero. There may be redundancy between the syntax elements cu_cbf / cu_coded_flag, tu_cbf_luma / tu_y_coded_flag, tu_cbf_cb / tu_cb_coded_flag and tu_cbf_cr / tu_cr_coded_flag.

[0549] 4. Example Techniques and Implementations

[0550] The detailed embodiments described below should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow sense. In addition, these embodiments can be combined in any way.

[0551] The method described below may also be applicable to other codecs that require signaling of certain syntax elements of chrominance color components (also known as dependent color components, such as B and R color components of an RGB color format).

[0552] 1. Syntax elements related to joint chroma residual coding can be signaled conditionally.

[0553] a. Syntax elements related to signaling of chroma delta QP / chroma QP offset (eg, pps_joint_cbcr_qp_offset) used in joint chroma residual codec may be conditionally signaled.

[0554] b. Syntax elements related to the indication of the use of joint chroma residual codec (eg, tu_cb_cr_joint_residual) may be conditionally signaled.

[0555] c. A syntax element may be signaled, which may be at the sequence / view / picture / slice / slice group level, to indicate whether joint chroma residual coding is enabled.

[0556] i. In one example, a flag may be signaled in a sequence parameter set (SPS) / video parameter set (VPS) / picture parameter set (PPS) / slice header / slice group header.

[0557] d. For the above example, the syntax elements related to joint chroma residual codec may be signaled under the condition that the chroma format is not equal to 4:0:0 and / or separate plane codec is disabled.

[0558] i. Alternatively, syntax elements related to joint chroma residual coding may be signaled under the condition that ChromaArrayType is not equal to a specific value such as 0.

[0559] ii. Alternatively, syntax elements related to joint chroma residual coding may be signaled conditional on ChromaArrayType and / or one or more syntax elements (eg, sequence level flags).

[0560] e. Alternatively, for the above example, under the condition that the chroma format is equal to 4:0:0 and / or separate plane coding is enabled, the signaling of syntax elements related to joint chroma residual coding can be skipped.

[0561] f. Alternatively, for the above example, in case ChromaArrayType is equal to a specific value such as 0, the signaling of syntax elements related to joint chroma residual coding can be skipped.

[0562] g. The above method can also be applied to other variants of the joint chroma residual coding and decoding method.

[0563] 2. Depending on the color format, the signaling of the indication of the use of the secondary transform may be skipped.

[0564] a. In one example, when the chroma format is not equal to 4:0:0 and / or separate plane coding is enabled, signaling of the indication of the use of the secondary transform (eg, st_idx) may be skipped.

[0565] b. In one example, when ChromaArrayType is equal to a specific value such as 0, signaling of an indication of the use of a secondary transform (eg, st_idx) may be skipped.

[0566] c. An indication of the use of secondary transform (eg, st_idx) may be signaled under the condition that ChromaArrayType is not equal to a specific value such as 0.

[0567] d. An indication of the use of secondary transform (eg, st_idx) may be signaled under the condition that the chroma format is not equal to 4:0:0 and / or separate plane codec is disabled.

[0568] 3. Syntax elements related to signaling of chroma deltaQP / chroma QP offset (eg, pps_cb_qp_offset and / or pps_cr_qp_offset) at picture level may be conditionally signaled.

[0569] a. In one example, when the chroma format is not equal to 4:0:0 and / or separate plane coding is enabled, the signaling of the chroma delta QP (eg, pps_cb_qp_offset and / or pps_cr_qp_offset) at the picture level may be skipped.

[0570] b. In one example, when ChromaArrayType is equal to a specific value such as 0, signaling of the picture-level chroma delta QP (eg, pps_cb_qp_offset and / or pps_cr_qp_offset) may be skipped.

[0571] c. Under the condition that ChromaArrayType is not equal to a specific value such as 0, the picture-level chroma delta QP (eg, pps_cb_qp_offset and / or pps_cr_qp_offset) may be signaled.

[0572] d. Under the condition that the chroma format is not equal to 4:0:0 and / or separate plane coding is disabled, the signaling of the chroma delta QP (eg, pps_cb_qp_offset and / or pps_cr_qp_offset) at the picture level may be signaled.

[0573] 4. Depending on the color format, the signaling of the indication of using luma-dependent chroma residual scaling (LDCRS) may be skipped.

[0574] a. In one example, when the chroma format is not equal to 4:0:0 and / or separate plane codec is enabled, signaling of an indication of the use of LDCRS (eg, slice_chroma_residual_scale_flag) may be skipped.

[0575] b. In one example, when ChromaArrayType is equal to a specific value such as 0, signaling of an indication of the use of LDCRS (eg, slice_chroma_residual_scale_flag) may be skipped.

[0576] c. Under the condition that ChromaArrayType is not equal to a specific value such as 0, an indication of the use of LDCRS (eg slice_chroma_residual_scale_flag) may be signaled.

[0577] d. Under the condition that the chroma format is not equal to 4:0:0 and / or separate plane codec is disabled, an indication of the use of LDCRS may be signaled (eg, slice_chroma_residual_scale_flag).

[0578] 5. Depending on the color format, the signaling of the bit depth of the samples of the chroma array and / or the indication of the value of the chroma quantization parameter range offset may be skipped.

[0579] a. In one example, when the chroma format is equal to 4:0:0 and / or separate plane coding is enabled, signaling of the bit depth of samples of the chroma array and / or an indication of the value of the chroma quantization parameter range offset (eg, bit_depth_chroma_minus8) may be skipped.

[0580] b. In one example, when ChromaArrayType is equal to a specific value such as 0, signaling of the bit depth of samples of the chroma array and / or an indication of the chroma quantization parameter range offset value (eg, bit_depth_chroma_minus8) may be skipped.

[0581] c. In one example, under the condition that ChromaArrayType is not equal to a specific value such as 0 or under the condition that (chroma format is not equal to 4:0:0 and / or separate plane coding is disabled), an indication of the value of the bit depth of samples of the chroma array and / or the chroma quantization parameter range offset (e.g., bit_depth_chroma_minus8) can be signaled.

[0582] 6. Depending on the color format, the signaling of the indication of the number of bits used to represent each PCM sample value of the chroma components may be skipped.

[0583] a. In one example, when the chroma format is not equal to 4:0:0 and / or separate plane codec is enabled, signaling of an indication of the number of bits used to represent each PCM sample value of the chroma component (eg, pcm_sample_bit_depth_chroma_minus1) may be skipped.

[0584] b. In one example, when ChromaArrayType is equal to a specific value such as 0, signaling of an indication of the number of bits used to represent each PCM sample value of the chroma component (eg, pcm_sample_bit_depth_chroma_minus1) may be skipped.

[0585] c. Under the condition that ChromaArrayType is not equal to a specific value such as 0, an indication of the number of bits used to represent each PCM sample value of the chroma component may be signaled (eg, pcm_sample_bit_depth_chroma_minus1).

[0586] d. Under the condition that the chroma format is not equal to 4:0:0 and / or separate plane coding is disabled, an indication of the number of bits used to represent each PCM sample value of the chroma component may be signaled (eg, pcm_sample_bit_depth_chroma_minus1).

[0587] 7. Syntax elements related to the ALF filter of the chroma component (e.g., a flag indicating whether the filter coefficients need to be sent, the filter coefficients, the APS index) can be conditionally signaled.

[0588] a. In one example, the condition is whether ChromaArrayType is not equal to a specific value such as 0 (eg, indicating a color format and / or whether there are any chroma components).

[0589] b. Depending on the color format being equal to 4:0:0 and / or separate plane coding being enabled, signaling of an indication of additional ALF chroma filter coefficients (eg, alf_chroma_filter_signal_flag, slice_alf_chroma_idc, slice_alf_aps_id_chroma) may be skipped.

[0590] c. Depending on the color format being equal to 4:0:0 and / or separate plane codec being enabled, signaling of the ALF chroma filter coefficients (eg, clipping flag / clipping parameters / filter coefficients) may be skipped.

[0591] d. Based on certain conditions, the temporal prediction of the ALF chroma filter can be disabled for the chroma codec block.

[0592] e. Alternatively, the relevant syntax elements may still be signaled, but inferred to default values ​​when the condition is true, such as alf_chroma_filter_signal_flag being inferred to be 0.

[0593] 8. Syntax elements related to the palette mode / intra-block copy (IBC) mode of chroma components (e.g., flags / mode indexes indicating whether palette / IBC is enabled for a chroma block / slice / picture / slice group / slice / brick) may be conditionally signaled.

[0594] a. In one example, the condition is whether ChromaArrayType is not equal to a specific value such as 0.

[0595] b. In one example, pred_mode_ibc_flag may be signaled conditionally.

[0596] c. Depending on the color format equal to 4:0:0 and / or separate plane codec being enabled, the signaling of the palette mode indication for the chroma components may be skipped.

[0597] d. Alternatively, the relevant syntax elements may still be signaled, but inferred to be default values ​​when the condition is true, e.g. the palette mode of the chroma components is inferred to be disabled.

[0598] 9. Syntax elements related to ALF filter time-domain prediction of chroma components (e.g., filter index / ALF APS index, on / off control flag) may be conditionally signaled.

[0599] a. In one example, the condition is whether ChromaArrayType is not equal to a specific value such as 0.

[0600] b. Depending on the color format being equal to 4:0:0 and / or separate plane codec being enabled, the signaling of the indication of the use of the secondary transform may be skipped.

[0601] c. Depending on the color format equal to 4:0:0 and / or separate plane coding being enabled, the signaling of the ALF filter time-domain prediction of the syntax elements related to the chroma components may be skipped.

[0602] d. Depending on certain conditions, the temporal prediction of the ALF chroma filter can be disabled for the chroma codec block.

[0603] 10. Those syntax elements in the SPS that are conditionally signaled according to the ChromaArrayType may be signaled with only one conditional check of the ChromaArrayType.

[0604] a. In one example, when ChromaArrayType is not equal to a specific value such as 0, some or all of the following syntax elements may also be signaled in a certain order.

[0605] i.qtbtt_dual_tree_intra_flag

[0606] ii.sps_cclm_enabled_flag

[0607] iii.pcm_sample_bit_depth_chroma_minus1

[0608] iv.bit_depth_chroma_minus8

[0609] b. In one example, the conditional check for ChromaArrayType can be replaced with a conditional check that the color format is equal to 4:0:0 and / or separate plane codec is enabled.

[0610] 11. For the above syntax elements, whether to invoke the decoding process of the chroma block may depend on the color format and / or the use of separate plane codecs.

[0611] a. For the above syntax elements, whether to call the decoding process of the chroma block may depend on the ChromaArrayType.

[0612] b. In one example, when ChromaArrayType is equal to a specific value such as 0, or the color format is 4:0:0 or separate plane coding is enabled, the ALF / secondary transform / chroma residual scaling process / quantization scaling matrix is ​​not applied to chroma blocks.

[0613] c. In one example, when ChromaArrayType is not equal to a specific value such as 0, ALF / secondary transform / chroma residual scaling process / quantization scaling matrix may be applied to chroma blocks.

[0614] 12. For the above syntax elements, they can still be signaled even when ChromaArrayType is equal to a specific value (e.g. 0) or the color format is 4:0:0 or separate plane coding is enabled. However, the decoding process and decoder shall ignore the decoded value.

[0615] a. In one example, when one of the above conditions is true, the above-mentioned syntax elements, such as those related to joint residual chroma coding (eg, pps_joint_cbcr_qp_offset), are not used in the decoding process and the decoder should ignore their values.

[0616] b. In one example, when one of the above conditions is true, the above-mentioned syntax elements, such as syntax elements related to chroma palette mode / chroma IBC mode / chroma ALF, are not used in the decoding process, and the decoder should ignore their values.

[0617] 13. For the above syntax elements, encoder constraints may be applied.

[0618] a. Conformant bitstreams shall satisfy: Joint chroma residual codec shall be disabled when the chroma format is 4:0:0 and / or separate plane codec is enabled.

[0619] b. The conforming bitstream shall satisfy: Chroma ALF shall be disabled when the chroma format is 4:0:0 and / or separate plane codec is enabled.

[0620] c. Conformant bitstreams shall satisfy: Chroma secondary transform (aka NSST / RST) shall be disabled (eg, sps_st_enabled_flag) when the chroma format is 4:0:0 and / or separate plane codec is enabled.

[0621] d. The conforming bitstream shall satisfy: when the chroma format is 4:0:0 and / or separate plane coding is enabled, the index (e.g., st_idx) of the chroma secondary transform matrix (aka, NSST / RST) shall be equal to a specific value such as 0.

[0622] e. The conforming bitstream shall satisfy: When the chroma format is 4:0:0 and / or separate plane coding is enabled, the chroma delta QP for joint chroma residual coding shall be equal to a specific value such as 0.

[0623] f. A conforming bitstream shall satisfy that when the chroma format is 4:0:0 and / or separate plane coding is enabled, the use of luma-dependent chroma residual scaling (eg, slice_chroma_residual_scale_flag) shall be equal to a specific value such as 0.

[0624] g. The conforming bitstream shall satisfy: when the chroma format is 4:0:0 and / or separate plane codec is enabled, the chroma delta QP signaled in the picture level shall be equal to a specific value such as 0.

[0625] h. When ChromaArrayType is equal to a specific value such as 0, the above syntax elements (eg, slice_chroma_residual_scale_flag) can still be decoded, but are not used in the decoding process and the decoder should ignore their values.

[0626] 14. Chroma color components can use different RST matrices.

[0627] a. In one example, separate indications of the RST matrix may be signaled for individual chroma components respectively.

[0628] i. In one example, the signaling of the indication of the RST matrix of the second chroma component may be dependent on the indication of the RST matrix of the first chroma component.

[0629] b. In one example, the RST matrix for the first chroma component may be signaled;

[0630] The matrix of the second chrominance component can be derived at the decoder side.

[0631] i. In one example, the matrix for the second chroma component can be derived from the RST matrix for a representative luma block.

[0632] ii. In one example, the matrix for the second chroma component may be derived from the RST matrix for the first chroma component, and the two matrices may not be equal.

[0633] iii. In one example, matrices for other chroma components may be derived from the chroma intra mode.

[0634] c. For more than one color component, how many indices need to be signaled may depend on the color subsampling format.

[0635] d. For more than one color component, how many indices need to be signaled may depend on whether dual tree is enabled.

[0636] e. For more than one color component, how many indices need to be signaled may depend on whether joint chroma residual coding is used.

[0637] i. In one example, an index may be signaled for residual coding.

[0638] 15. When joint chroma residual coding is applied to a block, the signaling of the secondary transform matrix index can be skipped.

[0639] a. In one example, when joint chroma residual codec is applied to a block, secondary transform is disabled for both of the two chroma components.

[0640] b. In one example, when joint chroma residual coding is applied to a block, secondary transform is enabled with a predefined matrix index.

[0641] i. In one example, when joint chroma residual coding is applied to one block, a secondary transform is applied to both color components with the same predefined matrix index.

[0642] ii. In one example, when joint chroma residual coding is applied to one block, a secondary transform is applied to two color components with different predefined matrix indices.

[0643] iii. In one example, when joint chroma residual coding is applied to one block, a secondary transform is applied to one color component with a predefined matrix index and the secondary transform is disabled for another color component.

[0644] c. Alternatively, when a block is coded with a secondary transform, the signaling of the use of joint chroma residual codec can be skipped.

[0645] i. Alternatively, furthermore, joint chroma residual coding is disabled for this block.

[0646] 16. When joint chroma residual coding is applied to a block, the signaling of the cross-component linear model (CCLM) method can be skipped.

[0647] a. In one example, when joint chroma residual codec is applied to a block, CCLM is disabled for both of the two chroma components.

[0648] b. Alternatively, when a block is coded with CCLM, the signaling of the use of joint chroma residual codec can be skipped.

[0649] i. Alternatively, furthermore, joint chroma residual coding is disabled for this block.

[0650] c. In one example, joint chroma residual coding and CCLM can be enabled at the same time.

[0651] The correlation between two chrominance residuals can be derived from the CCLM coefficients.

[0652] i. For example, assuming that Cb = a1*luma+b1 and Cr = a2*luma+b2, the correlation between the residuals of Cb and Cr (denoted as resiCb and resiCr, respectively) can be assumed to be: resiCb = a2*((resiCr–b1) / a1)+b2.

[0653] ii. Alternatively, the correlation between the residuals of Cb and Cr can be assumed to be: resiCb = a2*((resiCr–b1) / a1).

[0654] iii. Alternatively, the correlation between the residuals of Cb and Cr can be assumed to be: resiCb = a2*resiCr / a1.

[0655] iv. Alternatively, different correlations between the two chrominance residuals can be allowed, where their K (K>=1) can be derived from the CCLM coefficients.

[0656] d. In the above discussion, CCLM mode may refer to any type of mode that utilizes a cross-component linear model in VVC, such as LM mode, LM-T mode, and LM-L mode.

[0657] 17. When joint chroma residual coding is applied to a block, LDCRS can be disabled.

[0658] 18. Whether cu_cbf / cu_coded_flag is signaled may depend on the color format and / or component coding method (such as whether separate plane coding is enabled).

[0659] a. In one example, if the color format is 4:0:0 and / or separate plane codec is applied, cu_cbf / cu_coded_flag is not signaled.

[0660] i. If ChromaArrayType is equal to a specific value such as 0, cu_cbf / cu_coded_flag will not be signaled.

[0661] ii. Furthermore, alternatively, cu_cbf / cu_coded_flag is inferred to be 1 if not signaled when the color format is 4:0:0 and / or separate plane codec is applied.

[0662] 19. Whether to signal the coded block flag of the luma block (eg, tu_cbf_luma / tu_y_coded_flag) may depend on the color format and / or component coding method.

[0663] a. In one example, if the color format is 4:0:0 and / or separate plane coding is applied, the coded block flag (eg, tu_cbf_luma / tu_y_coded_flag) of the luma block is not signaled.

[0664] b. In one example, if the current block is coded in non-skipped inter mode and the color format is 4:0:0, the coded block flag (eg, tu_cbf_luma / tu_y_coded_flag) of the luma block is not signaled.

[0665] c. In one example, if the current block is coded in non-skipped inter mode and separate plane coding is applied, the coded block flag (eg, tu_cbf_luma / tu_y_coded_flag) of the luma block is not signaled.

[0666] d. In one example, if the current block is coded in non-skipped inter mode and ChromaArrayType is equal to a specific value such as 0, the coded block flag (eg, tu_cbf_luma / tu_y_coded_flag) of the luma block is not signaled.

[0667] 20. Whether tu_cbf_cb / tu_cb_coded_flag and tu_cbf_cr / tu_cr_coded_flag are signaled depends on the color format and / or component coding method (eg whether separate plane coding is enabled).

[0668] a. In one example, if the color format is 4:0:0 and / or separate plane codec is applied, tu_cbf_cb / tu_cb_coded_flag and tu_cbf_cr / tu_cr_coded_flag are not signaled.

[0669] b. In one example, if ChromaArrayType is equal to a specific value (eg, 0), tu_cbf_cb / tu_cb_coded_flag and tu_cbf_cr / tu_cr_coded_flag are not signaled.

[0670] 21. Whether to signal the codec block flag of a color component may depend on the codec block flags of other color components.

[0671] a. Whether tu_cbf_cr / tu_cr_coded_flag is signaled may depend on tu_cbf_luma / tu_y_coded_flag and / or tu_cbf_cb / tu_cb_coded_flag.

[0672] i. In one example, if tu_cbf_luma / tu_y_coded_flag and tu_cbf_cb / tu_cb_coded_flag are both equal to 0, then tu_cbf_cr / tu_cr_coded_flag is not signaled and is inferred to be 1. i.

[0673] b. Whether tu_cbf_cb / tu_cb_coded_flag is signaled may depend on tu_cbf_luma / tu_y_coded_flag and / or tu_cbf_cr / tu_cr_coded_flag.

[0674] i. In one example, if tu_cbf_luma / tu_y_coded_flag and tu_cbf_cr / tu_cr_coded_flag are both equal to 0, then tu_cbf_cb / tu_cb_coded_flag is not signaled and is inferred to be 1. i.

[0675] c. Whether tu_cbf_luma / tu_y_coded_flag is signaled may depend on tu_cbf_cb / tu_cb_coded_flag and / or tu_cbf_cr / tu_cr_coded_flag.

[0676] i. In one example, if tu_cbf_cb / tu_cb_coded_flag and tu_cbf_cr / tu_cr_coded_flag are both equal to 0, then tu_cbf_luma / tu_y_coded_flag is not signaled and is inferred to be 1. i.

[0677] d. The encoding and decoding order of the codec block flags for the three color components can be different from the codec flags for luma, Cb, and Cr.

[0678] e. The above method can be applied only when the current CU has only one TU and cu_cbf is equal to 1.

[0679] i. Alternatively, furthermore, the above method may be applied only when the current CU has multiple TUs but only one TU has non-zero coefficients (eg, SBT mode) and cu_cbf is equal to 1.

[0680] 22. Depending on whether the number of non-zero coefficients of the associated color component (such as numSigCoeff in Section 2.9.1) is greater than a threshold and the threshold may depend on the color format and / or component coding method (such as whether separate plane coding is enabled), it may be signaled whether to enable and / or how to apply RST to a block / coding unit (e.g., st_idx).

[0681] a. In one example, the threshold under a single tree codec structure may depend on whether the color format is 4:0:0 and / or whether separate plane codecs are applied.

[0682] i. For example, if the color format is 4:0:0 and / or separate plane codec is applied, the threshold is 1; otherwise, the threshold is 2.

[0683] b. In one example, the threshold value under the single-tree codec structure may depend on whether ChromaArrayType is equal to a specific value such as 0.

[0684] i. For example, if ChromaArrayType is equal to 0, the threshold is 1; otherwise, the threshold is 2.

[0685] 23. For the signaling of the RST side information (eg, st_idx) in the codec unit, it may depend only on the number of non-zero coefficients of the luma component, instead of all three color components.

[0686] a. In one example, if the number of non-zero coefficients of the luma block is greater than a threshold (eg, 1), the RST side information may be signaled.

[0687] b. In one example, when the luma block has all zero coefficients (eg, tu_cbf_luma / tu_y_coded_flag equals 0), no RST side information needs to be signaled and it is inferred that RST is disabled.

[0688] c. In one example, if one of the codec block flags of the three color components (eg, tu_cbf_luma / tu_y_coded_flag, tu_cbf_cb / tu_cb_coded_flag, and tu_cbf_cr / tu_cr_coded_flag) is equal to 0, no RST side information needs to be signaled, and it is inferred that RST is disabled.

[0689] d. When a single-tree codec structure is applied, the above method can be applied.

[0690] i. Alternatively, furthermore, when a single-tree codec structure is applied and the color format is not 4:0:0, the above method may be applied.

[0691] ii. Alternatively, furthermore, when a single-tree codec structure is applied, the color format is not 4:0:0 and separate plane codec is disabled, the above method may be applied.

[0692] 24. Signaling of RST side information (e.g., st_idx) for chroma components (such as in a separate tree codec structure), which may depend on the codec block flags of both chroma components.

[0693] a. In one example, if one of the codec block flags (eg, tu_cbf_cb / tu_cb_coded_flag or tu_cbf_cr / tu_cr_coded_flag) is equal to 0, the signaling of the RST side information may be skipped.

[0694] 25. Whether and / or how the indication of the SBT of a block is signaled may depend on the color format and / or component coding method (eg whether separate plane coding is enabled) and / or tu_cbf_luma / tu_y_coded_flag and / or tu_cbf_cb / tu_cb_coded_flag and / or tu_cbf_cr / tu_cr_coded_flag.

[0695] a. In one example, if tu_cbf_luma / tu_y_coded_flag, tu_cbf_cb / tu_cb_coded_flag, and tu_cbf_cr / tu_cr_coded_flag are all equal to 0, then no indication of SBT is signaled.

[0696] b. In one example, if tu_cbf_luma / tu_y_coded_flag is equal to 0, no indication of SBT is signaled.

[0697] c. In one example, if both tu_cbf_cb / tu_cb_coded_flag and tu_cbf_cr / tu_cr_coded_flag are equal to 0, then no indication of SBT is signaled.

[0698] d. In one example, the conditional signaling in the above bullet points may be applied only when the color format is 4:0:0 and / or separate plane codec is applied.

[0699] 26. Signaling of codec block flags for a TU (eg, tu_cbf_luma / tu_y_coded_flag and / or tu_cbf_cb / tu_cb_coded_flag and / or tu_cbf_cr / tu_cr_coded_flag) may depend on usage of the SBT of the CU including the TU.

[0700] a. In one example, if SBT is applied and a TU is not cleared by SBT, the tu_cbf_luma / tu_y_coded_flag of the TU is not signaled and is inferred to be 1.

[0701] b. In one example, if SBT is applied and a TU is not cleared by SBT, then the tu_cbf_cb / tu_cb_coded_flag and / or tu_cbf_cr / tu_cr_coded_flag of the TU is not signaled and is inferred to be 1.

[0702] c. The conforming bitstream shall satisfy: When SBT is enabled for a codec, the luma block shall have at least one non-zero coefficient (eg, tu_cbf_luma / tu_y_coded_flag is equal to 1).

[0703] i. Alternatively, the conforming bitstream shall satisfy: when SBT is enabled for the codec, one of the three color components shall have at least one non-zero coefficient (e.g., at least one of tu_cbf_luma / tu_y_coded_flag, tu_cbf_cb / tu_cb_coded_flag, tu_cbf_cr / tu_cr_coded_flag is equal to 1).

[0704] 27. The conforming bitstream shall satisfy: when the cu_cbf of the codec unit (e.g., for a non-skipped inter-coded block) is equal to 1, the luma block shall have at least one non-zero coefficient (e.g., tu_cbf_luma / tu_y_coded_flag is equal to 1).

[0705] a. Alternatively, the conforming bitstream shall satisfy: when the cu_cbf of the codec unit (e.g., for a non-skipped inter-coded block) is equal to 1, the three color components shall have at least one non-zero coefficient (e.g., at least one of tu_cbf_luma / tu_y_coded_flag, tu_cbf_cb / tu_cb_coded_flag, tu_cbf_cr / tu_cr_coded_flag is equal to 1).

[0706] 28. Regardless of the value of ChromaArrayType or the color format or the separate plane codec, the above syntax elements may still be signaled, but the decoded values ​​of those syntax elements may be ignored during decoding. That is, the decoding process may first check the value of ChromaArrayType or the color format or the separate plane codec before checking the decoded values ​​of those syntax elements.

[0707] 5. Additional Examples

[0708] 5.1 Example 1 of ALF decoding process

[0709] This embodiment shows an example of invoking the adaptive loop filter process, in which the chroma color format / separate plane codec flag (referred to by ChromaArrayType Indicates the condition.

[0710] The following suggested changes are underlined:

[0711] 8.8 Loop Filter Process

[0712] 8.8.1 Overview

[0713] Three loop filters, namely the deblocking filter, sample adaptive offset, and adaptive loop filter, are applied as specified by the following sequential steps:

[0714] For the deblocking filter, the following applies:

[0715] - Invoke the deblocking filter process specified in Section 8.8.2.1 with the reconstructed picture sample array S L , and when ChromaArrayType is not equal to 0, the array S Cb and S Cr As input, and with the modified reconstructed picture sample array S' L , and when ChromaArrayType is not equal to 0, the deblocked array S' Cb and S' Cr as output.

[0716] -Array S' L , and when ChromaArrayType is not equal to 0, the array S' Cb and S' Cr Assign to array S L , and when ChromaArrayType is not equal to 0, the array S Cb and S Cr (stands for decoded picture).

[0717] When sps_sao_enabled_flag is equal to 1, the following applies:

[0718] The sample adaptive offset process specified in Section 8.8.3.1 is called with the reconstructed picture sample array S L And when ChromaArrayType is not equal to 0, the array S Cb and S Cr As input, and using the modified reconstructed picture sample array S' L , when ChromaArrayType is not equal to 0, the array S' after sample adaptive offset Cb and S' Cr as output.

[0719] Array S' L , and when ChromaArrayType is not equal to 0, the array S' Cb and S' Cr Assign to array S L , and when ChromaArrayType is not equal to 0, the array S Cb and SCr (stands for decoded picture).

[0720] When sps_alf_enabled_flag is equal to 1, the following applies:

[0721] The adaptive loop filter process specified in Section 8.8.4.1 is called with the reconstructed picture sample array S L , When ChromaArrayType is not equal to 0, the array S Cb and S Cr As input, and with the modified reconstructed picture sample array S' L , And when ChromaArrayType is not equal to 0, After adaptive loop filtering Array S' Cb and S' Cr As output, .

[0722] Array S' L , And when ChromaArrayType is not equal to 0, the array S' Cb and S' Cr is assigned to the array S L , and And when ChromaArrayType is not equal to 0, They are Array S Cb and S Cr (stands for decoded picture).

[0723] 8.8.4 Adaptive Loop Filter Process

[0724] 8.8.4.1 Overview

[0725] The input to this process is the adaptive loop filter recPicture L The previous reconstructed image sample array, And when ChromaArrayType is not equal to 0, it is an array recPicture Cb and recPicture Cr .

[0726] The output of this process is the adaptive loop filter alfPicture L The modified reconstructed image sample array is then, And when ChromaArrayType is not equal to 0, it is an array alfPictureCb and alfPictureCr.

[0727] The modified reconstructed picture sample array alfPicture after the adaptive loop filter is respectively L 、alfPicture Cb and alfPicture Cr The sample values ​​in are initially set equal to the reconstructed picture sample array recPicture before the adaptive loop filter.L and recPicture Cb and recPicture Cr in the sample values.

[0728] When slice_alf_enabled_flag is equal to 1, for each coding tree unit with a luma coding tree block position (rx, ry) where rx = 0..PicWidthInCtbs-1 and ry = 0..PicHeightInCtbs-1, the following applies:

[0729] When alf_ctb_flag[0][rx][ry] is equal to 1, call the coding tree block filtering process for luma samples specified in Section 8.8.4.2, where recPicture L and alfPicture L and the luma coding tree block position (xCtb, yCtb) are set to be equal to (rx << CtbLog2SizeY, ry << CtbLog2SizeY) as inputs, and the output is the modified filtered picture alfPicture L .

[0730] When ChromaArrayType is not equal to 0 and alf_ctb_flag[1][rx][ry] is equal to 1, call the coding tree block filtering process for chroma samples specified in Section 8.8.4.4, where recPicture is set to be equal to recPicture Cb and alfPicture is set to be equal to alfPicture Cb and the chroma coding tree block position (xCtbC, yCtbC) is set to be equal to (rx << (CtbLog2SizeY-1), ry << (CtbLog2SizeY-1)) as an input, and the output is the modified filtered picture alfPicture Cb .

[0731] When ChromaArrayType is not equal to 0 and alf_ctb_flag[2][rx][ry] is equal to 1, call the coding tree block filtering process for chroma samples specified in Section 8.8.4.4, where recPicture is set to be equal to recPicture Cr and alfPicture is set to be equal to alfPicture Cr, and sets the chroma codec tree block position (xCtbC, yCtbC) equal to (rx<<(CtbLog2SizeY-1), ry<<(CtbLog2SizeY-1)) as input, and the output is the modified filter picture alfPicture Cr .

[0732] 7.3.5.1 General Strip Header Syntax

[0733]

[0734]

[0735] slice_alf_chroma_idc equal to 0 specifies that the adaptive loop filter is not applied to the Cb and Cr color components. slice_alf_chroma_idc equal to 1 indicates that the adaptive loop filter is applied to the Cb color component. slice_alf_chroma_idc equal to 2 indicates that the adaptive loop filter is applied to the Cr color component. slice_alf_chroma_idc equal to 3 indicates that the adaptive loop filter is applied to the Cb and Cr color components. When slice_alf_chroma_idc is not present, it is inferred to be equal to 0. When ChromaArrayType is equal to 0, slice_alf_chroma_idc is not used in the decoding process And does not exist.

[0736] -The maximum value maxVal of the truncated unary binary tu(v) is set equal to 3.

[0737] - slice_alf_aps_id_chroma specifies the adaptation_parameter_set_id to which the chroma components of the slice refer. When slice_alf_aps_id_chroma is not present, it is inferred to be equal to slice_alf_aps_id_luma[0]. The TemporalId of ALFAPS NAL units with adaptation_parameter_set_id equal to slice_alf_aps_id_chroma shall be less than or equal to the TemporalId of the coded slice NAL units.

[0738] - For intra slices and slices in IRAP pictures, slice_alf_aps_id_chroma shall not refer to the ALF APS associated with other pictures than the picture including the intra slice or IRAP picture.

[0739] - When ChromaArrayType is equal to 0, slice_alf_aps_id_chroma is not used in the decoding process And does not exist.

[0740] 5.2. Example 2 of Signaling of Chroma Delta QP

[0741] Added changes are underlined, and removed parts are marked with [[ ]].

[0742] 7.3.2.2. Picture parameter set RBSP syntax

[0743]

[0744]

[0745] pps_cb_qp_offset and pps_cr_qp_offset are used to derive Qp' Cb and Qp' Cr The brightness quantization parameter Qp' Y The values ​​of pps_cb_qp_offset and pps_cr_qp_offset should be in the range of -12 to +12 (inclusive). Does not exist When [[ChromaArrayType is equal to 0]], pps_cb_qp_offset and pps_cr_qp_offset Inferred to be 0 [[Not used during decoding; decoders should ignore their values]].

[0746] pps_slice_chroma_qp_offsets_present_flag equal to 1 indicates that the slice_cb_qp_offset and slice_cr_qp_offset syntax elements are present in the associated slice header. pps_slice_chroma_qp_offsets_present_flag equal to 0 indicates that these syntax elements are not present in the associated slice header. Does not exist When [[ChromaArrayType is equal to 0]], pps_slice_chroma_qp_offsets_present_flag should be equal to 0.

[0747] 7.3.5 Strip Header Syntax

[0748] 7.3.5.1 General Strip Header Syntax

[0749]

[0750]

[0751] 5.3 Example 3 of signaling for joint chroma residual coding

[0752] Added changes are underlined, and removed parts are marked with [[ ]].

[0753] 7.3.2.2 Picture parameter set RBSP syntax

[0754]

[0755]

[0756] 7.3.4.1 General slice group header syntax

[0757]

[0758] 7.3.6.12 Residual Codec Syntax

[0759]

[0760] 7.4.3.2 Picture Parameter Set RBSP Semantics

[0761] pps_joint_cbcr_qp_offset specifies the offset used to derive Qp′ CbCr The brightness quantization parameter Qp' Y The value of pps_joint_cbcr_qp_offset should be in the range of -12 to +12 (inclusive). Does not exist [[ChromaArrayType is equal to 0]], then pps_joint_cbcr_qp_offset is Inferred to be 0 [[Not used during decoding, decoders should ignore its value]].

[0762] 5.4 Example #4 on LMCS

[0763] Whether luma-dependent chroma residual scaling is enabled (e.g., slice_chroma_residual_scale_flag) depends on the ChromaArrayType. Added changes are underlined, and removed parts are marked with [[ ]].

[0764] 7.3.5 Strip Header Syntax

[0765] 7.3.5.1 General Strip Header Syntax

[0766]

[0767]

[0768] slice_chroma_residual_scale_flag equal to 1 specifies that chroma residual scaling is enabled for the current slice. slice_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling is not enabled for the current slice. When slice_chroma_residual_scale_flag is not present, it is inferred to be equal to 0. When ChromaArrayType is equal to 0, slice_chroma_residual_scale_flag does not exist because it is not used in the decoding process .

[0769] 5.5 Grouping Grammar Example #5

[0770] Added changes are underlined, and removed parts are marked with [[ ]].

[0771] 7.3.2.3 Sequence parameter set RBSP syntax

[0772]

[0773]

[0774]

[0775]

[0776]

[0777] 5.6 Example #6

[0778] Added changes are underlined, and removed parts are marked with [[ ]].

[0779] 7.3.5.3 Adaptive loop filter data syntax

[0780]

[0781]

[0782]

[0783]

[0784] alf_chroma_filter_signal_flag equal to 1 specifies that the chroma filter is signaled. alf_chroma_filter_signal_flag equal to 0 specifies that the chroma filter is not signaled. When ChromaArrayType is equal to 0, alf_ chroma_filter_signal_flag shall be equal to 0.

[0785] 5.7 Example #7

[0786] Added changes are underlined, and removed parts are marked with [[ ]].

[0787]

[0788]

[0789]

[0790] bit_depth_chroma_minus8 specifies the chroma array BitDepth C The bit depth of the sample and the chroma quantization parameter range offset QpBdOffset C The value is as follows:

[0791] BitDepth C =8+bit_depth_chroma_minus8 (7-5)

[0792] QpBdOffset C =6*bit_depth_chroma_minus8 (7-6)

[0793] bit_depth_chroma_minus8 should be in the range of 0 to 8 (inclusive). When ChromaArrayType is equal to When 0, bit_depth_chroma_minus8 is not used in the decoding process and decoders should ignore its value.

[0794] slice_alf_chroma_idc equal to 0 specifies that the adaptive loop filter is not applied to the Cb and Cr color components. slice_alf_chroma_idc equal to 1 specifies that the adaptive loop filter is applied to the Cb color component. slice_alf_chroma_idc equal to 2 specifies that the adaptive loop filter is applied to the Cr color component. slice_alf_chroma_idc equal to 3 specifies that the adaptive loop filter is applied to the Cb and Cr color components. When slice_alf_chroma_idc is not present, it is inferred to be equal to 0. When ChromaArrayType is equal to 0, slice_alf_chroma_idc is not used for decoding. is not present during the process and decoders should ignore its value.

[0795] The maximum value maxVal of the truncated unary binary tu(v) is set equal to 3.

[0796] slice_alf_aps_id_chroma specifies the adaptation_parameter_set_id to which the chroma components of the slice refer. When slice_alf_aps_id_chroma is not present, it is inferred to be equal to slice_alf_aps_id_luma[0]. The TemporalId of ALFAPS NAL units with adaptation_parameter_set_id equal to slice_alf_aps_id_chroma shall be less than or equal to the TemporalId of the coded slice NAL unit.

[0797] For intra slices and slices in IRAP pictures, slice_alf_aps_id_chroma shall not refer to the ALF APS associated with other pictures than the picture including the intra slice or the IRAP picture.

[0798] When ChromaArrayType is equal to 0, slice_alf_aps_id_chroma is not used in the decoding process It is not present, and decoders should ignore its value.

[0799] slice_chroma_residual_scale_flag equal to 1 specifies that chroma residual scaling is enabled for the current slice. slice_chroma_residual_scale_flag equal to 0 specifies that chroma residual scaling is not enabled for the current slice. When slice_chroma_residual_scale_flag is not present, it is inferred to be equal to 0. When ChromaArrayType is equal to 0, slice_chroma_residual_scale_flag does not exist because it is not used in the decoding process.

[0800] alf_chroma_filter_signal_flag equal to 1 specifies that the chroma filter is signaled. alf_chroma_filter_signal_flag equal to 0 specifies that the chroma filter is not signaled. When ChromaArrayType is equal to 0, alf_ chroma_filter_signal_flag shall be equal to 0.

[0801] 6. Example Implementations of the Disclosed Technology

[0802] Fig.161 is a block diagram of a video processing device 1600. Device 1600 can be used to implement one or more methods described herein. Device 1600 can be implemented in a smartphone, a tablet computer, a computer, an Internet of Things (IoT) receiver, etc. Device 1600 may include one or more processors 1602, one or more memories 1604, and video processing hardware 1606. (One or more) processors 1602 can be configured to implement one or more methods described in this document. (One or more) memories 1604 can be used to store data and code for implementing the methods and techniques described herein. Video processing hardware 1606 can be used to implement some of the techniques described in this document in hardware circuits, and can be partially or completely part of processor 1602 (e.g., a graphics processor core GPU or other signaling processing circuit).

[0803] In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. As defined by the grammar, the bitstream representation of the current video block may correspond, for example, to co-located bits within the bitstream or to extended bits in different positions. For example, a block may be encoded based on transformed and coded error residual values ​​and also using bits in a header and other fields in the bitstream. Here, a video block may be a logical unit corresponding to the processing operation being performed, such as a coding unit, a transform unit, a prediction unit, etc.

[0804] It will be appreciated that the disclosed methods and techniques will benefit video encoder and / or decoder embodiments incorporated within video processing apparatuses such as smartphones, laptops, desktop computers, and similar devices by enabling use of the techniques disclosed in this document.

[0805] Fig.17 is a flow chart of an example method 1700 of video processing. The method 1700 includes: at 1710, performing a conversion between a current video block and a bitstream representation of the current video block, wherein the conversion includes selectively enabling or disabling signaling of one or more of syntax flags associated with use of one or more of the following in response to detecting at least one condition: a joint chroma residual coding step, a luma-dependent chroma residual scaling (LDCRS) step, or an adaptive loop filter step.

[0806] Some embodiments may be described using the following clause-based format.

[0807] 1. A video processing method, comprising:

[0808] Converting is performed between a current video block and a bitstream representation of the current video block, wherein the converting includes selectively enabling or disabling signaling of one or more of syntax flags associated with use of one or more of the following in response to detecting at least one condition: a joint chroma residual coding step, a luma-dependent chroma residual scaling (LDCRS) step, a secondary transform step, a quantization step, or an adaptive loop filter step.

[0809] 2. A method as described in clause 1, wherein the one or more syntax flags are associated with any of the following: a chroma delta quantization parameter, a binary value indicating whether joint chroma residual coding and decoding is performed, the use of a secondary transform step, the number of bits used to represent the PCM sample values ​​of the chroma component, an index into the chroma secondary transform matrix, a binary value indicating whether joint chroma residual coding and decoding is performed, or a binary value indicating whether a dual-tree coding and decoding step is performed.

[0810] 3. A method as described in any of clauses 1-2, wherein one or more syntax flags correspond to the value of one or more of the following: pps_joint_cbcr_qp_offset, tu_cb_cr_joint_residual, st_idx, pps_cb_qp_offset, slice_chroma_residual_scale_flag, pcm_sample_bit_depth_chroma_minus1, qtbtt_dual_tree_intra_flag, sps_cclm_enabled_flag or sps_st_enabled_flag.

[0811] 4. A method as described in any of clauses 1-3, wherein one or more syntax flags are associated with a sequence, a view, a picture, a slice, a slice group, a codec block, a transform block, a prediction unit or other video data unit.

[0812] 5. A method as described in any of clauses 1-4, wherein at least one condition is associated with: the chroma format, the presence of a separate plane encoding and decoding step, the value of the ChromaArrayType variable, or the value of a flag in one or more syntax flags, or applying a secondary transformation to the current video block.

[0813] 6. The method according to any one of clauses 1 to 5, further comprising:

[0814] Based on the one or more syntax flags, a determination is made to apply a decoding step to the current video block.

[0815] 7. A video processing method, comprising:

[0816] A conversion is performed between a current video block and a bitstream representation of the current video block, wherein the conversion includes, in response to detecting at least one condition, selectively enabling or disabling signaling of one or more syntax flags associated with use of one or more of: a joint chroma residual coding step, a luma-dependent chroma residual scaling (LDCRS) step, a secondary transform step, a quantization step, or an adaptive loop filter step, wherein a first chroma component of the current video block is associated with a first reduced secondary transform (RST) matrix and a second chroma component of the current video block is associated with a second RST matrix.

[0817] 8. The method of clause 7, wherein the first flag is associated with a first RST matrix and the second flag is associated with a second RST matrix, wherein the first flag and the second flag are included in one or more syntax flags.

[0818] 9. The method of clause 7, wherein the first RST matrix is ​​transmitted in a bitstream representation and the second RST matrix is ​​derived.

[0819] 10. The method of clause 7, wherein the second marker is based at least in part on the first marker.

[0820] 11. The method of clause 7, wherein the second RST matrix is ​​derived from a representative luma block.

[0821] 12. The method of clause 7, wherein the second RST matrix is ​​derived from the first RST matrix, wherein the first RST matrix is ​​not equal to the second RST matrix.

[0822] 13. The method of clause 7, wherein the second RST matrix is ​​derived from a chroma intra mode.

[0823] 14. The method of clause 8, further comprising:

[0824] In response to determining to enable the joint chroma residual coding step for the current video block, signaling of the first RST flag and / or the second flag is disabled.

[0825] 15. The method of any one of clauses 7 to 14, further comprising:

[0826] In response to determining to enable a joint chroma residual coding step for the current video block, the first RST matrix and / or the second matrix is ​​applied.

[0827] 16. A method for video processing, comprising:

[0828] performing a conversion between a current video block and a bitstream representation of the current video block, wherein the conversion includes, in response to detecting at least one condition, selectively enabling or disabling signaling of one or more syntax flags associated with use of one or more of: a joint chroma residual coding step, a luma-dependent chroma residual scaling (LDCRS) step, a secondary transform step, a quantization step, or an adaptive loop filter step, wherein a first chroma component of the current video block is associated with a first reduced secondary transform (RST) matrix and a second chroma component of the current video block is associated with a second RST matrix; and

[0829] In response to determining to enable a joint chroma residual coding step for the current video block, signaling of a flag related to the use of CCLM is disabled on one or more chroma components of the current video block, wherein a flag in the one or more syntax flags is related to the use of a cross-component linear model (CCLM).

[0830] 17. The method of clause 16, further comprising:

[0831] In response to determining that the joint chroma residual coding step is enabled for the current video block, use of the CCLM is disabled on the current video block.

[0832] 18. The method of clause 16, further comprising:

[0833] In response to determining that the CCLM is applied to the current video block, a joint chroma residual encoding and decoding step is disabled on the current video block.

[0834] 19. The method of clause 16, further comprising:

[0835] Enables the use of CCLM and the use of the joint chroma residual encoding and decoding step on the current video block.

[0836] 20. The method of clause 16, wherein the joint chroma residual encoding and decoding step spans at least two chroma residual components, wherein a correlation between the two chroma residual components is based at least in part on coefficients associated with the CCLM.

[0837] 21. A method as described in any of clauses 16-20, wherein the use of CLLM includes the use of LM-T mode or LM-L mode.

[0838] 22. A method as described in any of clauses 1-21, wherein the one or more syntax flags are signaled individually or combined into a single flag.

[0839] 23. The method of any one of clauses 1 to 22, further comprising:

[0840] In response to determining that the joint chroma residual coding step is enabled, a luma dependent chroma residual scaling (LDCRS) step is disabled.

[0841] 24. A method as described in any of clauses 1-23, wherein one or more syntax flags are signaled individually or combined into a single flag.

[0842] 25. A method of video processing, comprising: making a decision for a current video block regarding selectively including one or more codec block flags in a bitstream representation of the current video block, and performing a conversion between the current video block and the bitstream representation of the current video block based on the decision, wherein the selectivity includes being based on a color format, a component codec method of the current video block, or a codec mode of the current video block.

[0843] 26. The method of clause 25, wherein a codec block flag (cu_cbf / cu_coded_flag) for a codec unit is not signaled when the color format is 4:0:0 or the component codec method is a separate plane codec method.

[0844] 27. The method of clause 25, wherein when the color format is 4:0:0 or the component coding method is a separate plane coding method, a codec block flag (tu_cbf_luma / tu_y_coded_flag) of the luma component of the current video block is not signaled.

[0845] 28. The method of clause 25, wherein when the color format is 4:0:0 and the current video block is coded in non-skipped inter mode, a coded block flag (tu_cbf_luma / tu_y_coded_flag) of the luma component of the current video block is not signaled.

[0846] 29. The method of clause 25, wherein when the color format is 4:0:0 or the component coding method is a separate plane coding method, a coding block flag (tu_cbf_cr / tu_cr_coded_flag or tu_cbf_cb / tu_cb_coded_flag) of one or more chroma components of the current video block is not signaled.

[0847] 30. The method of clause 25, wherein the signaling of the codec block flags (tu_cbf_cr / tu_cr_coded_flag or tu_cbf_cb / tu_cb_coded_flag) of one or more chrominance components of the current video block is based on the codec block flags (tu_cbf_luma / tu_y_coded_flag) of the luma component of the current video block.

[0848] 31. A method of video processing, comprising: making a decision to selectively apply a reduced secondary transform (RST) to the current video block for the current video block based on the number of non-zero coefficients in one or more color components of the current video block, and performing a conversion between the current video block and a bitstream representation of the current video block based on the decision.

[0849] 32. The method of clause 31, wherein making a determination further comprises comparing the number of non-zero coefficients to a threshold value that depends on a color format or a component codec method of the current video block.

[0850] 33. The method of clause 32, wherein the threshold is 1, wherein the color format is 4:0:0 or the component codec method is a separate plane codec method.

[0851] 34. The method of clause 32, wherein the threshold is 2, wherein the color format is other than 4:0:0, and wherein the component coding method is other than a separate plane coding method.

[0852] 35. The method of clause 31, wherein the selective application is based only on the number of non-zero coefficients in the luma component of the current video block.

[0853] 36. A method of video processing, comprising: performing conversion between a current video block and a bitstream representation of the current video block, wherein the bitstream representation is based on selectively including syntax elements indicating information about chroma coding based on characteristics of chroma.

[0854] 37. A method as described in clause 36, wherein the information about the chroma codec includes bit depth information for chroma array samples and / or values ​​of chroma quantization parameter range offsets, and wherein the characteristics of the chroma include the chroma format.

[0855] 38. The method of clause 36, wherein the information about chroma coding comprises information about adaptive loop filtering used during conversion, and wherein the characteristic of chroma corresponds to whether a ChromaArrayType syntax element is equal to a particular value.

[0856] 39. The method of clause 36, wherein the information about chroma codec includes information about adaptive loop filtering used during conversion, and wherein the characteristic of chroma corresponds to whether the chroma format is 4:0:0.

[0857] 40. The method of clause 36, wherein the information about chroma coding comprises information about a palette mode or an intra block copy mode of chroma, and wherein the characteristic of chroma corresponds to whether a ChromaArrayType syntax element is equal to a specific value.

[0858] Further examples and embodiments of clauses 36 to 40 are provided in items 5, 7, 8 and 12 of the previous section.

[0859] 41. An apparatus in a video system, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement the method of any one of clauses 1 to 40.

[0860] 42. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for performing the method of any one of clauses 1 to 40.

[0861] Fig. 22 2200 is a block diagram illustrating an exemplary video processing system 2200 in which various techniques disclosed herein may be implemented. Various embodiments may include some or all of the components of system 2200. System 2200 may include an input 2202 for receiving video content. The video content may be received in a raw or uncompressed format (e.g., 8 or 10-bit multi-component pixel values), or may be in a compressed or encoded format. Input 2202 may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces (such as Ethernet, passive optical networks (PONs), etc.) and wireless interfaces (such as Wi-Fi or cellular interfaces).

[0862] System 2200 may include a codec component 2204 that implements various codecs or encoding methods described in this document. Codec component 2204 can reduce the average bit rate of the video from input 2202 to the output of codec component 2204 to generate a codec representation of the video. Therefore, codec technology is sometimes referred to as video compression or video transcoding technology. As represented by component 2206, the output of codec component 2204 can be stored or sent via the connected communication. The bitstream (or codec) representation of the storage or communication of the video received at input 2202 can be used by component 2208 to generate pixel values ​​or displayable video sent to display interface 2210. The process of generating user-visible video from the bitstream representation is sometimes referred to as video decompression. In addition, although some video processing operations are referred to as "codec" operations or tools, it should be understood that codec tools or operations are used at the encoder, and the decoder will perform the corresponding decoding tools or operations of the inverse codec results.

[0863] Examples of peripheral bus interfaces or display interfaces may include Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) or Displayport, etc. Examples of storage interfaces include SATA (Serial Advanced Technology Attachment), PCI, IDE interfaces, etc. The techniques described in this document may be implemented in various electronic devices, such as mobile phones, laptop computers, smart phones, or other devices capable of performing digital data processing and / or video display.

[0864] Fig.23 23 is a flowchart representation of a method 2300 of video decoding according to the present technology. The method 2300 includes, at operation 2310, determining based on a rule whether one or more syntax flags applicable to a region of the video are included in a syntax structure of a codec representation of the video. The one or more syntax flags indicate a joint chroma residual encoding and decoding step for representing the region in the codec representation. The method 2300 also includes, at operation 2320, generating one or more decoded video blocks for the region by parsing the codec representation based on the presence or absence of the one or more syntax flags in the codec representation.

[0865] Fig.24 24 is a flowchart representation of a method 2400 for video encoding according to the present technology. The method 2400 includes, at operation 2410, conditionally encoding one or more syntax flags applicable to a region of the video in a syntax structure of a codec representation of the video based on a rule. The one or more syntax flags indicate a joint chroma residual encoding step for representing the region in the codec representation.

[0866] In some embodiments, the rule is indicated based on the chroma format of the region of the video. In some embodiments, determining the chroma format includes determining whether the chroma component is encoded and decoded separately. In some embodiments, when the chroma component is encoded and decoded separately, one or more syntax flags are not present in the codec representation. In some embodiments, when the chroma component is not encoded and decoded separately, one or more syntax flags are present in the codec representation.

[0867] In some embodiments, the rule indicates that in the case where the chroma format is 4:0:0, one or more syntax flags are not present in the codec representation. In some embodiments, the rule indicates that in the case where the chroma format is not 4:0:0, one or more syntax flags are present in the codec representation. In some embodiments, the rule indicates that the chroma format is indicated by the variable ChromaArrayType in the codec representation. In some embodiments, the rule indicates that in the case where the variable ChromaArrayType is equal to a predetermined value, one or more syntax flags are not present in the codec representation. In some embodiments, the rule indicates that in the case where the variable ChromaArrayType is not equal to a predetermined value, one or more syntax flags are present in the codec representation. In some embodiments, the predetermined value is 0.

[0868] In some embodiments, the region comprises a sequence. In some embodiments, a flag is signaled in a sequence parameter set. In some embodiments, the region comprises a picture. In some embodiments, a plurality of flags are signaled in a picture parameter set, the plurality of flags being associated with a quantization parameter offset. In some embodiments, the region comprises a transform unit. In some embodiments, a flag associated with the transform unit is associated with use of a joint chroma residual encoding and decoding step. In some embodiments, the region comprises a slice. In some embodiments, the region comprises a view or a slice group.

[0869] Fig.25 is a flow chart representation of a method 2500 of video decoding according to the present technology. The method 2500 includes, at operation 2510, determining a chroma format for a region of a video. The method 2500 includes, at operation 2520, determining whether one or more syntax flags applicable to the region of the video are included in a syntax structure of a codec representation of the video based on the chroma format. The one or more syntax flags indicate the use of quantization parameter offsets for representing the region in the codec representation. The method 2500 also includes, at operation 2530, generating one or more decoded video blocks for the region of the video by parsing the codec representation based on the presence or absence of the one or more syntax flags.

[0870] Fig.26 is a flow chart representation of a method 2600 of video decoding according to the present technology. The method 2600 includes, at operation 2610, determining a chroma format for a region of a video. The method 2600 includes, at operation 2620, determining, based on the chroma format, that one or more syntax flags are absent from a syntax structure of a codec representation of the video, the one or more syntax flags indicating use of a secondary transform applicable to the video region. The secondary transform is applied between a dequantization step and an inverse primary transform. The method 2600 also includes, at operation 2630, generating one or more decoded video blocks for the video region by parsing the codec representation based on the absence of the one or more syntax flags.

[0871] In some embodiments, one or more decoded video blocks are generated based on the determination of the chroma format. In some embodiments, the method includes, if one or more syntax flags are present in the codec representation, discarding one or more syntax flags used to generate the one or more decoded video blocks.

[0872] Fig. 27 is a flowchart representation of a method 2700 of video encoding according to the present technology. The method 2700 includes, at operation 2710, determining a chroma format associated with a region of a video. The method 2700 also includes, at operation 2720, conditionally encoding one or more syntax flags in a syntax structure of a codec representation of the video based on the chroma format. The one or more syntax flags indicate that use of a quantization parameter offset for representing a region in the codec representation is enabled.

[0873] Fig.28 is a flowchart representation of a method 2800 of video encoding according to the present technology. The method 2800 includes, at operation 2810, determining a chroma format of a region of a video. The method 2800 also includes, at operation 2820, generating a codec representation of the video by encoding the region based on the chroma format without including one or more syntax flags indicating use of a secondary transform in a syntax structure of the codec representation of the video. The secondary transform is applied between the forward primary transform and the quantization step.

[0874] In some embodiments, determining the chroma format comprises determining whether color components of the chroma format are separately encoded or decoded.In some embodiments, the region comprises a picture.

[0875] In some embodiments, one or more syntax flags are not present in the codec representation when (1) the chroma format is 4:0:0 or (2) the color components of the chroma format are separately coded. In some embodiments, one or more syntax flags are present in the codec representation when (1) the chroma format is not 4:0:0 or (2) the color components of the chroma format are not separately coded.

[0876] In some embodiments, the chroma format is indicated by a variable ChromaArrayType in the bitstream representation. In some embodiments, when the variable ChromaArrayType is equal to a predetermined value, one or more syntax flags are not present in the codec representation. In some embodiments, when the variable ChromaArrayType is not equal to a predetermined value, one or more syntax flags are present in the codec representation. In some embodiments, the predetermined value is 0.

[0877] In some embodiments, the level corresponding to the region comprises a sequence level. In some embodiments, the determination of the chroma format comprises a single step of determining a variable ChromaArrayType. In some embodiments, in the case where the variable ChromaArrayType is not equal to a predetermined value, one or more syntax flags are sorted in the codec representation. In some embodiments, the predefined value is 0. In some embodiments, the one or more syntax flags comprise at least one of (1) qtbtt_dual_tree_intra_flag, (2) sps_cclm_enabled_flag, or (3) pcm_sample_bit_depth_chroma_minus1.

[0878] In some embodiments, one or more decoded video blocks are generated without applying an adaptive loop filter, a secondary transform, a chroma residual scaling step, or a quantization scaling matrix to the blocks of the video when (1) the variable ChromaArrayType is equal to a predetermined value, (2) the chroma format is 4:0:0, or (3) the color components of the chroma format are encoded and decoded separately. In some embodiments, generating the one or more decoded video blocks includes applying an adaptive loop filter, a secondary transform, a chroma residual scaling step, or a quantization scaling matrix to the blocks of the video when (1) the variable ChromaArrayType is not equal to a predetermined value.

[0879] In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are individually coded and decoded, the joint chroma residual coding step is indicated as disabled in the codec representation. In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are individually coded and decoded, the chroma adaptive loop filtering process is indicated as disabled in the codec representation. In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are individually coded and decoded, the chroma secondary transform is indicated as disabled in the codec representation. In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are individually coded and decoded, the index of the chroma secondary transform matrix is ​​set to a predetermined value in the codec representation. In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are individually coded and decoded, the chroma quantization parameter offset used in the joint chroma residual coding step is set to a predetermined value in the codec representation. In some embodiments, when the chroma format is 4:0:0 or the color components of the chroma format are encoded separately, the use of luma-dependent chroma residual scaling is set to a predetermined value in the codec representation. In some embodiments, when the chroma format is 4:0:0 or the color components of the chroma format are encoded separately, the picture-level chroma quantization parameter offset is set to a predetermined value in the codec representation. In some embodiments, the predetermined value is 0.

[0880] Fig.29A 29 is a flowchart representation of a method 2900 for video encoding according to the present technology. The method 2900 includes, at operation 2910, determining whether one or more syntax flags applicable to the video region are missing in a syntax structure of a codec representation of the video based on the chroma format of the video including the region. The one or more syntax flags indicate the use of a luma-dependent chroma residual scaling codec step for representing the region in the codec representation. The luma-dependent chroma residual scaling codec step includes a multiplication process that compensates luma signaling of the video with chroma signaling of the video. The method 2900 includes, at operation 2920, generating one or more decoded video blocks of the video region by parsing the codec representation according to the determination.

[0881] Fig.29Bis a flowchart representation of a method 2950 for video encoding according to the present technology. The method 2950 includes, at operation 2960, determining a chroma format of a video including a region. The method 2950 includes, at operation 2970, generating a codec representation of the video based on the chroma format by encoding the region without including one or more syntax flags in a syntax structure of the codec representation of the video, wherein the one or more syntax flags indicate that a luma-dependent chroma residual scaling codec step is used to represent the region in the codec representation, wherein the luma-dependent chroma residual scaling codec step includes a multiplication process that compensates luma signaling of the video with chroma signaling of the video.

[0882] Fig. 30A 3000 is a flowchart representation of a method 3000 of video encoding according to the present technology. The method 3000 includes, at operation 3010, determining that one or more syntax flags applicable to the region of the video are not present in a syntax structure of a codec representation of the video based on a chroma format of the video including the region. The one or more syntax flags indicate a number of bits used to represent each pulse codec modulation sample value of the chroma component. The method 3000 includes, at operation 3020, generating one or more decoded video blocks of the video region by parsing the codec representation according to the determination.

[0883] Fig. 30B is a flowchart representation of a method 3050 for video encoding according to the present technology. The method 3050 includes, at operation 3060, determining a chroma format of a video including a region. The method 3050 also includes, at operation 3070, generating a codec representation of the video based on the chroma format by encoding the region without including one or more syntax flags in a syntax structure of the codec representation of the video, wherein the one or more syntax flags indicate a number of bits used to represent each pulse codec modulation sample value of a chroma component.

[0884] Fig.31A is a flowchart representation of a method 3100 of video encoding according to the present technology. The method 3100 includes, at operation 3110, determining whether one or more syntax flags applicable to the region of the video are included in a syntax structure of a codec representation of the video based on the chroma format of the video including the region. The one or more syntax flags are associated with an adaptive loop filter (ALF) time domain prediction step for the chroma component. The method 3100 also includes, at operation 3120, generating one or more decoded video blocks of the video region by parsing the codec representation according to the determination.

[0885] Fig.31B31 is a flow chart of a method 3150 of video encoding according to the present technology. The method 3150 includes, at operation 3160, determining a chroma format of a video including a region. The method 3150 also includes, at operation 3170, conditionally encoding one or more syntax flags in a syntax structure of a codec representation of the video based on the chroma format, wherein the one or more syntax flags are associated with an adaptive loop filter (ALF) temporal prediction step of a chroma component.

[0886] In some embodiments, one or more decoded video blocks are generated based on the determination of the chroma format.In some embodiments, the method includes discarding one or more syntax flags used to generate the one or more decoded video blocks if one or more syntax flags are present in the codec representation.

[0887] In some embodiments, determining the chroma format includes determining whether color components of the chroma format are encoded and decoded separately. In some embodiments, the region includes a stripe. In some embodiments, when (1) the chroma format is 4:0:0, or (2) the color components of the chroma format are encoded and decoded separately, one or more syntax flags are not present in the codec representation. In some embodiments, when (1) the chroma format is not 4:0:0, or (2) the color components of the chroma format are not encoded and decoded separately, one or more syntax flags are present in the codec representation. In some embodiments, the chroma format is indicated by a variable ChromaArrayType in the bitstream representation. In some embodiments, when the variable ChromaArrayType is equal to a predetermined value, one or more syntax flags are not present in the codec representation. In some embodiments, when the variable ChromaArrayType is not equal to a predetermined value, one or more syntax flags are present in the codec representation. In some embodiments, the predetermined value is 0.

[0888] In some embodiments, the one or more syntax flags include at least one of: (1) a filter index, (2) an adaptive loop filter (ALF) adaptive parameter set (APS) index, or (3) a control flag. In some embodiments, the level corresponding to the region includes a sequence level. In some embodiments, the determination of the chroma format includes a single step of determining a variable ChromaArrayType. In some embodiments, when the variable ChromaArrayType is not equal to a predefined value, the one or more syntax flags are sorted in the codec representation. In some embodiments, the predefined value is 0. In some embodiments, the one or more syntax flags include at least one of: (1) qtbtt_dual_tree_intra_flag, (2) sps_cclm_enabled_flag, or (3) pcm_sample_bit_depth_chroma_minus1.

[0889] In some embodiments, one or more decoded video blocks are generated without applying an adaptive loop filter, a secondary transform, a chroma residual scaling step, or a quantization scaling matrix to the blocks of video when (1) the variable ChromaArrayType is equal to a predetermined value, (2) the chroma format is 4:0:0, or (3) the color components of the chroma format are encoded and decoded separately. The secondary transform is applied between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform.

[0890] In some embodiments, generating one or more decoded video blocks includes applying an adaptive loop filter, a secondary transform, a chroma residual scaling step, or a quantization scaling matrix to the video block if (1) ChromaArrayType is not equal to a predetermined value. The secondary transform is applicable between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform.

[0891] In some embodiments, in case the chroma format is 4:0:0 or the color components of the chroma format are individually coded, the joint chroma residual coding step is indicated as disabled in the codec representation. The joint chroma residual coding step comprises determining a joint residual which is the average of the residuals associated with the chroma components.

[0892] In some embodiments, the chroma adaptive loop filtering process is indicated as disabled in the codec representation when the chroma format is 4:0:0 or the color components of the chroma format are encoded separately. In some embodiments, the chroma secondary transform is indicated as disabled in the codec representation when the chroma format is 4:0:0 or the color components of the chroma format are encoded separately. The chroma secondary transform is applied between the forward primary transform and the quantization step or between the dequantization step and the inverse primary transform.

[0893] In some embodiments, when the chroma format is 4:0:0 or the color components of the chroma format are encoded and decoded separately, the index of the chroma secondary transform matrix is ​​set to a predetermined value in the codec representation. The chroma secondary transform is applicable between the forward primary transform and the quantization step or between the dequantization step and the inverse primary transform.

[0894] In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are encoded separately, the chroma quantization parameter offset used in the joint chroma residual encoding and decoding step is set to a predetermined value in the codec representation. The joint chroma residual encoding and decoding step includes determining a joint residual, which is the average of the residuals associated with the chroma components. In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are encoded separately, the use of the luminance-dependent chroma residual scaling encoding and decoding step is set to a predetermined value in the codec representation. In some embodiments, in the case where the chroma format is 4:0:0 or the color components of the chroma format are encoded separately, the chroma quantization parameter offset at the picture level is set to a predetermined value in the codec representation. In some embodiments, the predetermined value is 0.

[0895] Fig.32 32 is a flowchart representation of a method 3200 of video processing according to the present technology. The method 3200 includes, at operation 3210, for conversion between a block of video and a bitstream representation of the video, determining a plurality of reduced secondary transform (RST) matrices corresponding to a plurality of chrominance color components of the block. The secondary transform is applicable between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform. The method 3200 also includes, at operation 3220, performing a conversion based on the determination.

[0896] In some embodiments, each of a plurality of RST matrices is signaled for a corresponding chroma color component in the bitstream representation.In some embodiments, the signaling of a second RST matrix for a second chroma color component is based on the signaling of a first RST matrix for a first chroma color component.

[0897] In some embodiments, the plurality of RST matrices include a first RST matrix and a second RST matrix, the first RST matrix is ​​signaled in the bitstream representation, and the second RST matrix is ​​not present in the bitstream representation. In some embodiments, the second RST matrix is ​​derived based on the RST matrix for the luma block. In some embodiments, the second RST matrix is ​​derived based on the first RST matrix, and the first RST matrix and the second RST matrix are different. In some embodiments, the second RST matrix is ​​derived based on a chroma intra mode encoding and decoding step.

[0898] In some embodiments, subsets of multiple RST matrices are signaled in the bitstream representation. In some embodiments, the number of subsets of multiple RST matrices is determined based on the color subsampling format of the block. In some embodiments, the number of subsets of multiple RST matrices is determined based on whether a dual-tree encoding and decoding step is enabled for the block. In some embodiments, the number of subsets of multiple RST matrices is determined based on whether a joint chroma residual encoding and decoding step is used. In some embodiments, a single RST matrix is ​​signaled in the bitstream representation when a joint chroma residual encoding and decoding step is used.

[0899] Fig.33 is a flowchart representation of a method 3300 of video processing according to the present technology. The method 3300 includes, at operation 3310, for a conversion between a block of video and a bitstream representation of the video, determining that one or more matrix indices for a secondary transform are not present in the bitstream representation when a joint chroma residual encoding and decoding step is applied to the block. The secondary transform is applied to the block between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform. The joint chroma residual encoding and decoding step includes determining a joint residual, which is an average of residuals associated with chroma components. The method 3300 also includes, at operation 3320, performing a conversion based on the determination.

[0900] In some embodiments, a secondary transform is disabled for a chroma component of the block. In some embodiments, the secondary transform is applied using one or more predefined matrix indices. In some embodiments, the same predefined matrix index is used for the chroma components of the block. In some embodiments, different predefined matrix indices are applied to different chroma components of the block. In some embodiments, a predefined matrix index is used for a first chroma component of the block, and wherein the secondary transform is disabled for a second chroma component of the block.

[0901] Fig.34 is a flowchart representation of a method 3400 of video processing according to the present technology. The method 3400 includes, at operation 3410, for a conversion between a block of video and a bitstream representation of the video, determining that a syntax flag indicating the use of a joint chroma residual encoding and decoding step is missing in the bitstream representation when a secondary transform is applied to the block between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform. The joint chroma residual encoding and decoding step includes determining a joint residual, which is an average of residuals associated with chroma components. The method 3400 also includes, at operation 3420, performing a conversion based on the determination. In some embodiments, the joint chroma residual encoding and decoding step is disabled for the block.

[0902] Fig.35is a flowchart representation of a method 3500 of video processing according to the present technology. The method 3500 includes, at operation 3510, for a conversion between a block of a video and a bitstream representation of the video, determining that a syntax flag indicating the use of a cross-component linear model encoding and decoding step is missing in the bitstream representation when a joint chroma residual encoding and decoding step is applied to the block. The joint chroma residual encoding and decoding step includes determining a joint residual, which is an average of the residuals associated with the chroma components. The method 3500 also includes, at operation 3520, performing a conversion based on the determination. In some embodiments, the cross-component linear model encoding and decoding step is disabled for the chroma components of the block.

[0903] Fig.36 is a flowchart representation of a method 3600 of video processing according to the present technology. The method 3600 includes, at operation 3610, for a conversion between a block of a video and a bitstream representation of the video, determining that a syntax flag indicating the use of a joint chroma residual encoding and decoding step is missing in the bitstream representation when a cross-component linear model is applied to the block. The joint chroma residual encoding and decoding step includes determining a joint residual, which is an average of residuals associated with chroma components. The method 3600 also includes, at operation 3620, performing a conversion based on the determination. In some embodiments, the joint chroma residual encoding and decoding step is disabled for the block.

[0904] Fig.37 37 is a flowchart representation of a method 3700 of video processing according to the present technology. The method 3700 includes, at operation 3710, for converting between a block of video and a bitstream representation of the video, determining a correlation between two chroma residuals used in a joint chroma residual encoding and decoding step based on coefficients of a cross-component linear model encoding and decoding step for the block. The joint chroma residual encoding and decoding step includes determining a joint residual, which is an average of the residuals associated with the chroma components. The method 3700 includes, at operation 3720, performing a conversion based on the determination.

[0905] In some embodiments, the first chrominance component is Cb, the second chrominance component is Cr, a1, a2, b1 and b2 are coefficients of the cross-component linear model encoding and decoding step, and Cb=a1*luma+b1 and Cr=a2*luma+b2. In some embodiments, the correlation between the first residual resiCb of the first chrominance component and the second residual resiCr of the second chrominance component is expressed as resiCb=a2*(resiCr-b1) / a1)+b2. In some embodiments, the correlation between the first residual resiCb of the first chrominance component and the second residual resiCr of the second chrominance component is expressed as resiCb=a2*(resiCr-b1) / a1). In some embodiments, the correlation between the first residual resiCb of the first chrominance component and the second residual resiCr of the second chrominance component is expressed as resiCb=a2*resiCr / a1.

[0906] In some embodiments, the method includes determining one or more additional correlations between two chroma residuals based on coefficients of a cross-component linear model encoding and decoding step. In some embodiments, the cross-component linear model encoding and decoding step includes a linear model (LM) mode, a LM-T mode, or a LM-L mode.

[0907] Fig.38 38 is a flowchart representation of a method 3800 of video processing according to the present technology. The method 3800 includes, at operation 3810, for a conversion between a video block and a bitstream representation of the video, determining to disable a luma-dependent chroma residual scaling codec step when a joint chroma residual codec step is applied to the block. The luma-dependent chroma residual scaling codec step includes a multiplication process that compensates luma signaling of the video with chroma signaling of the video. The method 3800 also includes, at operation 3820, performing the conversion based on the determination.

[0908] Fig.39 39 is a flowchart representation of a method 3900 of video processing according to the present technology. The method 3900 includes, at operation 3910, for conversion between a block of a video and a codec representation of the video, determining based on rules related to codec characteristics of the video whether a syntax flag associated with the block indicating the presence of at least one non-zero coefficient is included in a syntax structure in the codec representation. The method 3900 also includes, at operation 3920, performing the conversion based on the determination.

[0909] In some embodiments, the codec characteristics include a color format of the video. In some embodiments, the codec characteristics include using component codec techniques to represent the video in a codec representation. In some embodiments, the component codec techniques include separately encoding and decoding color components of the video.

[0910] In some embodiments, the syntax structure is for a codec unit. In some embodiments, the rules specify that in the case where the color format is 4:0:0 or the color components of the video are encoded separately, the syntax flag is omitted in the syntax structure. In some embodiments, the color format is indicated by the variable ChromaArrayType in the codec representation, and wherein the rules specify that the syntax flag is omitted if ChromaArrayType is equal to a predefined value. In some embodiments, in the case where the syntax flag is omitted in the syntax structure, the codec unit is considered to be encoded with at least one non-zero coefficient.

[0911] In some embodiments, the block is divided into one or more transform units, and wherein the syntax structure is for the transform unit of the block. In some embodiments, the block belongs to the luminance component of the video. In some embodiments, the rule specifies that the syntax flag is omitted in the syntax structure when the color format is 4:0:0 or the color component of the block is encoded and decoded separately. In some embodiments, the rule specifies that the syntax flag is omitted in the syntax structure when the color format is 4:0:0 and the block is encoded and decoded using the inter-frame mode codec tool without using the skip mode codec. In some embodiments, the rule specifies that the syntax flag is omitted in the codec representation when the color component of the block is encoded and decoded separately and the block is encoded and decoded using the inter-frame mode codec tool without using the skip mode codec. In some embodiments, the color format is indicated by the variable ChromaArrayType in the codec representation, and wherein the rule specifies that there is no syntax flag when ChromaArrayType is equal to a predefined value and the block is encoded and decoded using the inter-frame mode codec tool without using the skip mode codec.

[0912] In some embodiments, the block belongs to a chroma component of a video. In some embodiments, the rules specify that if the color format is 4:0:0 or the color components of the block are encoded and decoded separately, no syntax flags are present in the syntax structure. In some embodiments, the color format is indicated by the variable ChromaArrayType in the codec representation, and the rules specify that if ChromaArrayType is equal to a predefined value, no syntax flags are present in the syntax structure. In some embodiments, the predefined value is 0.

[0913] In some embodiments, the rule specifies whether the syntax flag is present in the syntax structure based on one or more syntax flags of other blocks belonging to other components of the video. In some embodiments, the block belongs to the Cr component of the video, and one or more syntax flags are used for other blocks belonging to the Cb component and / or the luminance component of the video. In some embodiments, in the case where one or more syntax flags indicate that other blocks are not coded with at least one non-zero coefficient, the syntax flag is omitted in the syntax structure, and the block is considered to be coded with at least one non-zero coefficient. In some embodiments, the block belongs to the Cb component of the video, and one or more syntax flags are used for other blocks belonging to the Cr component and / or the luminance component of the video. In some embodiments, in the case where one or more syntax flags indicate that other blocks are not coded with at least one non-zero coefficient, the syntax flag is omitted in the syntax structure, and the block is considered to be coded with at least one non-zero coefficient. In some embodiments, the block belongs to the luminance component of the video, and one or more syntax flags are used for other blocks belonging to the Cr component and / or the Cb component of the video. In some embodiments, in the case where one or more syntax flags indicate that other blocks are not coded with at least one non-zero coefficient, the syntax flag is omitted in the syntax structure, and the block is considered to be coded with at least one non-zero coefficient.

[0914] In some embodiments, the first order in which the block and the other blocks are processed is different from the second order in which the syntax flag and the one or more other syntax flags are arranged in the codec representation. In some embodiments, the rule may apply to the following cases: the block includes only one transform unit, and the block is indicated as being encoded with at least one non-zero coefficient by a second syntax flag in the codec unit syntax structure. In some embodiments, the rule may apply to the following cases: the block has multiple transform units, and only one of the multiple transform units has a non-zero coefficient for a sub-block transform, and the block is indicated as being encoded with at least one non-zero coefficient by a second syntax flag in the codec unit syntax structure.

[0915] In some embodiments, the rule specifies that whether to include a syntax flag in the syntax structure is based on the use of a sub-block transform for a block that includes a transform unit. In some embodiments, the block belongs to a luma component of the video, and in the case where the sub-block transform is applied to a portion of the block instead of the transform unit, the syntax flag is omitted in the syntax structure, and the block is considered to be encoded and decoded with at least one non-zero coefficient. In some embodiments, the block belongs to a chroma component of the video, and in the case where the sub-block transform is applied to a portion of the block instead of the transform unit, the syntax flag is omitted in the syntax structure, and the block is considered to be encoded and decoded with at least one non-zero coefficient. In some embodiments, the block belongs to a luma component of the video, and in the case where the sub-block transform is enabled for the block, the syntax flag present in the syntax structure indicates that the block is encoded and decoded with at least one non-zero coefficient. In some embodiments, the block belongs to one of a luma component, a Cr component, or a Cb component of the video, and the syntax flag present in the syntax structure indicates that the block is encoded and decoded with at least one non-zero coefficient when the sub-block transform is enabled for the block.

[0916] In some embodiments, the block belongs to a luma component of a video, and wherein, in the case where a first syntax flag in the codec unit syntax structure indicates that the block is encoded with at least one non-zero coefficient, a second syntax flag in the transform unit syntax structure indicates that the block is encoded with at least one non-zero coefficient. In some embodiments, the block belongs to one of a luma component, a Cr component, or a Cb component of a video, and in the case where a first syntax flag in the codec unit syntax structure indicates that the block is encoded with at least one non-zero coefficient, a second syntax flag in the transform unit syntax structure indicates that the block is encoded with at least one non-zero coefficient.

[0917] Fig.40 4000 is a flowchart representation of a method 4000 of video processing according to the present technology. The method 4000 includes, at operation 4010, for a conversion between a block of a video and a codec representation of the video, determining information about a secondary transform with reduced dimensionality based on rules associated with codec characteristics of the video. The secondary transform is applicable between a forward primary transform and a quantization step or between a dequantization step and an inverse primary transform, and the reduced dimensionality is reduced based on the dimensionality of the block. The method 4000 also includes, at operation 4020, performing the conversion according to the determination.

[0918] In some embodiments, the information about the secondary transform includes whether the secondary transform is enabled for the block. In some embodiments, the information about the secondary transform includes a manner in which the secondary transform is applied to the block. In some embodiments, the information about the secondary transform specifies a secondary transform kernel applicable to the block.

[0919] In some embodiments, the codec characteristics include a color format of the video. In some embodiments, the codec characteristics include using component codec techniques to represent the video in a codec representation. In some embodiments, the component codec techniques include separately encoding and decoding color components of the video.

[0920] In some embodiments, a block is encoded and decoded using a single tree codec structure, and the threshold is based on whether the color format is 4:0:0 or the color components of the video are encoded and decoded separately. In some embodiments, the threshold is 1 when the color format is 4:0:0 or the color components are encoded and decoded separately. In some embodiments, if the color format is not 4:0:0 and the color components are encoded and decoded together, the threshold is 2. In some embodiments, the color format is indicated by the variable ChromaArrayType in the codec representation. In some embodiments, the threshold is 1 when ChromaArrayType is equal to a predefined value. In some embodiments, the threshold is 2 when ChromaArrayType is not equal to a predefined value. In some embodiments, the predefined value is 0.

[0921] In some embodiments, the rule specifies that whether to include information about the secondary transform in the codec representation is based solely on the number of non-zero coefficients associated with a block belonging to the luma component of the video. In some embodiments, the information about the secondary transform is included in the codec representation if the number of non-zero coefficients associated with the block is greater than a threshold. In some embodiments, the threshold is 1. In some embodiments, the information about the secondary transform is omitted from the codec representation if the number of non-zero coefficients associated with the block is 0 and the secondary transform is considered disabled for the block.

[0922] In some embodiments, the rules specify that information about the secondary transform is omitted in the codec representation when a syntax flag of the block indicates that the block is not coded with at least one non-zero coefficient. The block belongs to a Cr component, a Cb component, or a luminance component of a video, and the secondary transform is considered disabled for the block. In some embodiments, the rules apply to the case where the block is coded using a single-tree codec structure. In some embodiments, the color format of the video is not 4:0:0. In some embodiments, the color components of the video are coded separately. In some embodiments, the rules specify that whether to include information about the secondary transform in the codec representation is based on the syntax flag of the block. The block belongs to a chroma component of the video. In some embodiments, the block belongs to a chroma component of the video, and when a syntax flag in the transform unit syntax indicates that the block is not coded with at least one non-zero coefficient, information about the secondary transform is omitted in the codec representation. In some embodiments, the codec characteristics include the color format of the video, the use of a component codec technique for indicating the video in the codec representation, or a syntax flag in a transform unit syntax structure indicating whether the block is coded with at least one non-zero coefficient. In some embodiments, the component codec technique includes coding the color components of the video separately.

[0923] In some embodiments, the block belongs to a Cr component, a Cb component, or a luma component of a video, and the rule specifies that information about the secondary transform is omitted in the codec representation when a syntax flag indicates that the block is not coded with at least one non-zero coefficient and other syntax flags in the transform unit syntax structure indicate that other blocks belonging to other components of the video are not coded with at least one non-zero coefficient. In some embodiments, the block belongs to a luma component of a video, and the rule specifies that information about the secondary transform is omitted in the codec representation when a syntax flag indicates that the block is not coded with at least one non-zero coefficient. In some embodiments, the block belongs to a Cr component or a Cb component, and the rule specifies that information about the secondary transform is omitted in the codec representation when a syntax flag indicates that the block is not coded with at least one non-zero coefficient and a second syntax flag in the transform unit syntax structure indicates that a second block belonging to another chroma component of the video is not coded with at least one non-zero coefficient. In some embodiments, the rule applies to the case where the color format is 4:0:0 or the color components are coded separately. In some embodiments, the block includes samples associated with multiple color components.

[0924] Fig.4141 is a flowchart representation of a method 4100 of video processing according to the present technology. The method 4100 includes, at operation 4110, for conversion between a video and a codec representation of the video, determining based on a condition whether a syntax element associated with an adaptive loop filtering (ALF) operation of a chroma component of the video is included in the codec representation of the video. The method 4100 includes, at operation 4120, performing the conversion based on the determination.

[0925] In some embodiments, the syntax element comprises a flag indicating whether at least coefficients for an adaptive loop filtering operation are included in the codec representation. In some embodiments, the syntax element is signaled in an adaptive parameter set (APS) corresponding to the ALF operation. In some embodiments, the codec representation comprises a plurality of syntax elements related to filter coefficients for an adaptive loop filtering operation.

[0926] In some embodiments, the syntax element comprises an index for an adaptation parameter set, the adaptation parameter set comprising information for one or more adaptive loop filters. In some embodiments, the syntax element is signaled in a slice header.

[0927] In some embodiments, the syntax element includes an indication of whether ALF operation is enabled for any chroma component.In some embodiments, the syntax element is signaled in a slice header.

[0928] In some embodiments, whether the color format of the video is present is indicated by a variable ChromaArrayType in the codec representation, and the condition includes whether ChromaArrayType is not equal to a predefined value. In some embodiments, the predefined value is 0. In some embodiments, the syntax element includes at least alf_chroma_filter_signalflag, slice_alf_chroma_idc, or slice_alf_aps_id_chroma.

[0929] In some embodiments, the syntax element includes at least a clipping flag, a clipping parameter, or a filter coefficient. In some embodiments, the syntax element is omitted in the codec representation if a condition is met. In some embodiments, the condition specifies that: there is no chroma component, the color format of the video is 4:0:0, or the color components of the video are encoded and decoded separately. In some embodiments, the syntax element is signaled with a codec value in the codec representation if the condition is met, and the actual value of the syntax element is set to a default value regardless of the codec value. In some embodiments, the syntax element includes alf_chroma_filter_signal_flag, and the default value is 0.

[0930] In some embodiments, temporal prediction of adaptive loop filtering operations is disabled based on a condition. In some embodiments, a syntax element indicates temporal prediction of adaptive loop filtering operations, and the syntax element is omitted in the codec representation if a condition is met. The condition specifies that the color format of the video is 4:0:0 or the color components of the video are encoded and decoded separately.

[0931] Fig.42 4200 is a flowchart representation of a method 4200 of video processing according to the present technology. The method 4200 includes, at operation 4210, for conversion between a video and a codec representation of the video, determining based on a condition whether the codec representation includes a syntax element associated with a codec technology for a chroma component of the video. The method 4200 includes, at operation 4220, performing the conversion based on the determination.

[0932] In some embodiments, the codec technique includes at least: a palette mode codec technique in which a palette representing pixel values ​​is used to represent sample values ​​of a video block; or an intra block copy codec mode codec technique. In some embodiments, the syntax element includes a flag or index indicating whether the codec technique is enabled for the chroma component. In some embodiments, the syntax element includes pred_mode_ibc_flag.

[0933] In some embodiments, the color format of the video is represented by a variable ChromaArrayType in the codec representation, where the condition includes whether ChromaArrayType is not equal to a predefined value. In some embodiments, the predefined value is 0. In some embodiments, syntax elements associated with the palette mode codec technique are omitted in the codec representation if the condition is met. The condition specifies that the color format of the video is 4:0:0 or the color components of the video are encoded and decoded separately.

[0934] In some embodiments, the syntax element is signaled with a codec value in the codec representation if the condition is met, and the actual value of the syntax element is set to a default value regardless of the codec value. In some embodiments, the syntax element includes a flag indicating the use of the palette mode codec technique, and the default value is 0, indicating that the palette mode codec technique is disabled.

[0935] Fig.43 43 is a flowchart representation of a method 4300 of video processing according to the present technology. The method 4300 includes, at operation 4310, for conversion between a video and a codec representation of the video, determining whether a syntax element indicating a characteristic of a chroma component of the video is included in the codec representation based on a condition associated with a color format of the video. The method 4300 includes, at operation 4320, performing the conversion based on the determination.

[0936] In some embodiments, the characteristic comprises a sample bit depth of a chroma array of the video. In some embodiments, the characteristic comprises a chroma quantization parameter range offset. In some embodiments, the syntax element comprises bit_depth_chroma_minus8. In some embodiments, the syntax element comprises pps_joint_cbcr_qp_offset.

[0937] In some embodiments, the syntax element is omitted in the codec representation if the condition that the color format of the specified video is 4:0:0 or the color components of the video are encoded and decoded separately is met. In some embodiments, the color format of the video is indicated by the variable ChromaArrayType in the codec representation, and the syntax element is omitted in the codec representation if the condition that the specified ChromaArrayType is equal to a predefined value is met. In some embodiments, the syntax element is included in the codec representation if the condition that the color format of the specified video is not 4:0:0 or the color components of the video are not encoded and decoded separately is met. In some embodiments, the color format of the video is indicated by the variable ChromaArrayType in the codec representation, and the syntax element is included in the codec representation if the condition that the specified ChromaArrayType is not equal to a predefined value is met. In some embodiments, the predefined value is 0.

[0938] In some embodiments, a syntax element is included in the codec representation if a condition is met, the condition specifies that the color format of the video is 4:0:0, the color components of the video are encoded and decoded separately, or a variable ChromaArrayType indicating the color format of the video is equal to a predefined value. The syntax element is ignored during the conversion process. In some embodiments, at least one other syntax element associated with whether the condition is met is processed before processing or ignoring the syntax element.

[0939] In some embodiments, performing the conversion includes generating a bitstream representation based on the blocks of the video. In some embodiments, performing the conversion includes generating the blocks of the video from the bitstream representation.

[0940] The disclosed and other solutions, examples, embodiments, modules and functional operations described in this document may be implemented in digital electronic circuits, or in computer software, firmware or hardware, including the structures disclosed in this document and their structural equivalents, or a combination of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, for example, one or more computer program instruction modules encoded on a computer-readable medium, for execution by a data processing device or for controlling the operation of the data processing device. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a storage device, a combination of substances that implement machine-readable propagation signaling, or a combination of one or more of them. The term "data processing device" covers all devices, apparatuses and machines for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, a device may also include code that creates an execution environment for the mentioned computer program, for example, code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. Propagated signaling is artificially generated signaling, for example, machine-generated electrical, optical or electromagnetic signaling, which is generated to encode information for transmission to a suitable receiver device.

[0941] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on a single computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0942] The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by performing operations on input data and generating output. The processes and logic flows can also be performed by, and devices can also be implemented as, special purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0943] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, the computer will also include or be operatively coupled to receive data from or transfer data to one or more mass storage devices (e.g., magnetic, magneto-optical or optical disks) for storing data, or both. However, the computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and storage devices, including, for example, semiconductor storage devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into a dedicated logic circuit.

[0944] Although the patent document includes many details, these details should not be interpreted as limitations on any subject matter or the scope of the claimed protection, but rather as descriptions of features of specific embodiments for specific technologies. Certain features described in the patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments or in any suitable sub-combination, respectively. Moreover, although the features may be described above as working in certain combinations and even initially claimed as such, one or more features in the claimed combination may be cut out from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variation of the sub-combination.

[0945] Similarly, while operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve the desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0946] Only a few implementations and examples are described, and other implementations, enhancements, and variations may be made based on what is described and illustrated in this patent document.

Claims

1. A method for processing video data, comprising: for conversion between a video region of a video and a bitstream of the video, determining, based on a chroma format of the video, whether at least one syntax element associated with an adaptive loop filtering operation of a chroma component of the video region is included in the bitstream; as well as performing said converting according to said determining, wherein a first temporal identifier of a first network abstraction layer unit of an adaptive parameter set used for the adaptive loop filtering operation of the chrominance component of the video region is less than or equal to a second temporal identifier of a second network abstraction layer unit of the video region, and an adaptive parameter set identifier included in the adaptive parameter set is the same as an index of the adaptive parameter set referenced by the chrominance component of the video region, wherein the at least one syntax element comprises a plurality of syntax elements associated with filter coefficients of the adaptive loop filtering operation of the chrominance component of the video region, and The multiple syntax elements are included in the adaptive parameter set corresponding to the adaptive loop filtering operation.

2. The method according to claim 1, wherein: The at least one syntax element also includes a flag indicating whether at least one coefficient of the adaptive loop filtering operation of the chroma component of the video region is included in the bitstream, and the flag is included in the adaptive parameter set corresponding to the adaptive loop filtering operation of the chroma component of the video region.

3. The method according to claim 2, wherein: In response to the flag not being present in the bitstream, the value of the flag is inferred to be equal to a default value, indicating that none of the at least one coefficient of the adaptive loop filtering operation for the chroma component of the video region is included in the bitstream.

4. The method according to claim 1, wherein: The at least one syntax element further comprises an index to the adaptation parameter set, and wherein the index is included in a slice header or a picture header, and the adaptation parameter set comprises information of one or more adaptive loop filters.

5. The method according to claim 1, wherein: The at least one syntax element also includes a first indication indicating whether the adaptive loop filtering operation is enabled for any chroma component.

6. The method according to claim 5, wherein: The first indication is included in a slice header.

7. The method according to claim 1, wherein: The at least one syntax element further comprises a second indication related to filter time domain prediction of the adaptive loop filtering operation of the chroma component.

8. The method according to claim 1, wherein: In a case where the chroma format is not equal to a specific value, the at least one syntax element is included in the bitstream.

9. The method according to claim 8, wherein: The specific value is 4:0:

0.

10. The method according to claim 1, wherein: The converting includes encoding the video region into the bitstream.

11. The method according to claim 1, wherein: The converting includes decoding the video region from the bitstream.

12. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to: For conversion between a video region of a video and a bitstream of the video, determining, based on a chroma format of the video, whether at least one syntax element associated with an adaptive loop filtering operation of a chroma component of the video region is included in the bitstream; and performing said converting according to said determining, in, The first temporal identifier of the first network abstraction layer unit of the adaptive parameter set used by the adaptive loop filtering operation of the chrominance component of the video region is less than or equal to the second temporal identifier of the second network abstraction layer unit of the video region, and the adaptive parameter set identifier included in the adaptive parameter set is the same as the index of the adaptive parameter set referenced by the chrominance component of the video region, wherein the at least one syntax element comprises a plurality of syntax elements associated with filter coefficients of the adaptive loop filtering operation of the chrominance component of the video region, and The multiple syntax elements are included in the adaptive parameter set corresponding to the adaptive loop filtering operation.

13. The device according to claim 12, wherein: In case the chroma format is not equal to 4:0:0, including the at least one syntax element in the bitstream, The at least one syntax element further includes an index of the adaptive parameter set, wherein the index is included in a slice header or a picture header and in the adaptive parameter set, and the adaptive parameter set includes information of one or more adaptive loop filters.

14. The device according to claim 13, wherein: The at least one syntax element further comprises a flag, the flag indicating whether at least one coefficient of the adaptive loop filtering operation of the chroma component of the video region is included in the bitstream, and the flag is included in the adaptive parameter set corresponding to the adaptive loop filtering operation of the chroma component of the video region, and, wherein, in response to the flag not being present in the bitstream, the value of the flag is inferred to be equal to a default value, indicating that none of the at least one coefficient of the adaptive loop filtering operation for the chrominance component of the video region is included in the bitstream.

15. A non-transitory computer-readable storage medium storing instructions that cause a processor to: For conversion between a video region of a video and a bitstream of the video, determining, based on a chroma format of the video, whether at least one syntax element associated with an adaptive loop filtering operation of a chroma component of the video region is included in the bitstream; and performing said converting according to said determining, in, The first temporal identifier of the first network abstraction layer unit of the adaptive parameter set used by the adaptive loop filtering operation of the chrominance component of the video region is less than or equal to the second temporal identifier of the second network abstraction layer unit of the video region, and the adaptive parameter set identifier included in the adaptive parameter set is the same as the index of the adaptive parameter set referenced by the chrominance component of the video region, wherein the at least one syntax element comprises a plurality of syntax elements associated with filter coefficients of the adaptive loop filtering operation of the chrominance component of the video region, and The multiple syntax elements are included in the adaptive parameter set corresponding to the adaptive loop filtering operation.

16. The non-transitory computer-readable storage medium of claim 15, wherein: In case the chroma format is not equal to 4:0:0, including the at least one syntax element in the bitstream, The at least one syntax element further includes an index of the adaptive parameter set, wherein the index is included in a slice header or a picture header and in the adaptive parameter set, and the adaptive parameter set includes information of one or more adaptive loop filters.

17. The non-transitory computer-readable storage medium of claim 15, wherein: The at least one syntax element further comprises a flag, the flag indicating whether at least one coefficient of the adaptive loop filtering operation of the chroma component of the video region is included in the bitstream, and the flag is included in the adaptive parameter set corresponding to the adaptive loop filtering operation of the chroma component of the video region, and, wherein, in response to the flag not being present in the bitstream, the value of the flag is inferred to be equal to a default value, indicating that none of the at least one coefficient of the adaptive loop filtering operation for the chrominance component of the video region is included in the bitstream.

18. A method for storing and determining a bit stream of a video, comprising: determining, for a video region of a video, based on a chroma format of the video, whether at least one syntax element associated with an adaptive loop filtering operation of a chroma component of the video region is included in the bitstream; generating the bitstream according to the determination; as well as storing the bitstream in a non-transitory computer-readable storage medium, wherein a first temporal identifier of a first network abstraction layer unit of an adaptive parameter set used for the adaptive loop filtering operation of the chrominance component of the video region is less than or equal to a second temporal identifier of a second network abstraction layer unit of the video region, and an adaptive parameter set identifier included in the adaptive parameter set is the same as an index of the adaptive parameter set referenced by the chrominance component of the video region, wherein the at least one syntax element comprises a plurality of syntax elements associated with filter coefficients of the adaptive loop filtering operation of the chrominance component of the video region, and The multiple syntax elements are included in the adaptive parameter set corresponding to the adaptive loop filtering operation.

Images