Scaling the coefficients based on position

Abstract

To provide a video image processing method that performs a conversion of a video image block of a video image and a bit stream expression of the video image, by using a scaling matrix.SOLUTION: A method is for determining whether or not a scaling matrix is adopted on the basis of the application of a secondary conversion matrix to one part of a video image block of a video image in accordance with a rule, and includes the determination of the use of the scaling matrix scaling at least some coefficients of the video image block, wherein the secondary conversion matrix is used for converting at least some residual coefficients of a portion of the video image block during the conversion, and the conversion of the video image block of the video image and a bit stream expression of the video image by using the selected scaling matrix.SELECTED DRAWING: Figure 18

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a divisional application of Japanese Patent Application No. 2022-511034, which is the national phase application of International Patent Application No. PCT / CN2020 / 110229 filed on August 20, 2020. This application claims priority to and the benefit of International Patent Application No. PCT / CN2019 / 101555, filed on August 20, 2019. All of the above-mentioned patent applications are incorporated herein by reference in their entireties.

[0002] This patent document relates to video coding techniques, devices and systems. [Background technology]

[0003] Despite advances in video compression, digital video is still used widely across the Internet and It is the largest bandwidth user of any other digital communications network. As the number of connected user devices capable of displaying and viewing digital content increases, Bandwidth demands for video usage are predicted to continue to increase. Summary of the Invention

[0004] Regarding digital video coding, in particular, scaling and / or transformation matrices Video encoding and decoding devices, systems, and methods for use herein.

[0005] In one exemplary embodiment, a method of image processing is disclosed. The method includes: converting between the blocks and a coded representation of the image, the coded representation being in a format The format rule is a conversion skip mode for the video block. the applicability of the code is determined by the coding condition of the video block; The format rule is such that a syntax element representing availability of the conversion skip mode is the transform skip mode is omitted from the encoded representation before encoding to the encoded representation skipping the application of a forward transform to at least some of the coefficients in the decoding Sometimes, an inverse transform is applied to at least some of the coefficients before decoding from the encoded representation. and skipping the step of

[0006] In another exemplary embodiment, a method of image processing is disclosed. The method includes: Forward and inverse transforms are used to convert between adjacent image blocks and the coded representation of the image. Depending on which conversion is used for the transformation, either an in-loop filter or a post-reconstruction filter is used. determining whether to use a forward transform to transform the encoded representation; skipping applying the forward transform to at least some of the coefficients before and, upon decoding, performing at least some of the inverse transform before decoding from the encoded representation. skipping the application of the in-loop filter or reconstruction to some coefficients; and performing the conversion based on the use of a post-filter.

[0007] In another exemplary aspect, a method of video processing is disclosed. The method includes: For conversion between the image block and the encoded representation of the image, the encoding mode of the image block is determined based on the encoding mode of the image block. Based on this, determine the scaling tool multiple and convert using the scaling tool. and performing a scaling tool to scale the image block during encoding. or, upon decoding, scaling at least some of the coefficients representing this encoding Descaling at least some of the coefficients from the representation.

[0008] In another exemplary aspect, a method of video processing is disclosed. The method includes: A block for said conversion of said video blocks for conversion between a block and a coded representation of a video. Differential Pulse Code Modulation (BDPCM) encoding tool or Quantized Residual BDPCM (QR -BDPCM) Determining that the use of the scaling tool is disabled due to the encoding tool and performing the conversion without using a scaling tool, The use of a scaling tool scales at least some of the coefficients that represent a video block during encoding. or descaling at least some of the coefficients from the coded representation upon decoding. This includes

[0009] In another exemplary aspect, a method of video processing is disclosed. The method includes: For conversion between the image and the coded representation of the image, a conversion matrix is ​​selected for the conversion. selecting a scaling matrix based on the scaling matrix, is used to scale at least some coefficients of the block, said transformation matrix being used to transform at least some coefficients of the image block during the transformation. The method includes: selecting a first pixel and performing a transformation using a scaling matrix.

[0010] In another exemplary aspect, a method of image processing is disclosed. The method comprises: , based on whether a secondary transformation matrix is ​​applied to some of the video blocks of the video. , determining whether to apply a scaling matrix, the scaling matrix being: used to scale at least some of the coefficients of the video block; The retransform matrix transforms at least some residual coefficients of the portions of the video block during the transformation. determining a scaling matrix to be used for the image; and converting between image blocks and a bitstream representation of the video.

[0011] In another exemplary aspect, a method of image processing is disclosed. For a video block having a shape similar to that of the video block, a relationship between the video block and the encoded representation of the video is determining a scaling matrix for use in a transformation of said encoded representation; A syntax element that signals the scaling matrix, the scaling matrix being the transformation sometimes used to scale at least some of the coefficients of the video block. , determining performing the transformation based on the scaling matrix.

[0012] In another exemplary aspect, a method of video processing is disclosed. The method includes: 1. A method of video processing comprising converting between a clock and an encoded representation of the video, the method comprising: The image block is transformed based on a rule to which the scaling matrix is ​​applied. A first number of positions and a second number of positions to which no scaling matrix is ​​applied during the transformation. nothing.

[0013] In another exemplary aspect, a method of video processing is disclosed. The method includes: determining whether a scaling matrix is ​​applied when converting between the image and the encoded representation of the image; And, performing the transformation based on the scaling matrix, The coded representation indicates the number of elements of the scaling matrix, the number indicating the coefficients to be zeroed. It depends on whether it is applied to the coefficients of the image block.

[0014] In another exemplary aspect, a method of image processing is disclosed. The method comprises: 2. A method of video processing comprising: converting between video blocks of a video and a coded representation of said video. There was, The image block is transformed by applying a K×L transformation matrix to transform the coefficients of the image block. In the coded representation after zeroing all the transform coefficients except for the top left M×N transform coefficient, Expressed, The encoded representation is a representation of the scaling matrix elements at positions corresponding to the zeroing. configured to eliminate signaling; The scaling matrix is ​​used to scale the transform coefficients.

[0015] In another exemplary aspect, a method of video processing is disclosed. The method includes: When converting between a block and a coded representation of the video, a single determining whether to use a quantization matrix of size determining that all video blocks having a quantization matrix performing the transform using the quantization matrix.

[0016] In another exemplary aspect, a method of video processing is disclosed. The method includes: for converting between the coded representation of the block and the video block based on coding mode information; determining whether a transform skip mode is enabled for the transform; and performing the conversion based on the determining, wherein the conversion skip mode wherein, during the transform, a transform is applied to at least some coefficients representing the video block. Skip using

[0017] In another exemplary aspect, another method of video processing is disclosed, the method comprising: Block Differential Pulse Code Modulation (BDPCM) or Quantized Residual BDPCM (QR - Due to the use of the BDPCM mode, the coded representation of the video block and the relationship between the video block determining that a scaling matrix is ​​to be used for the transformation; performing the transformation using a scaling matrix, the scaling matrix being is used to scale at least some of the coefficients representing

[0018] In another exemplary aspect, another method of video processing is disclosed, the method comprising: Block Differential Pulse Code Modulation (BDPCM) or Quantized Residual BDPCM (QR - Due to the use of the BDPCM mode, the coded representation of the video block and the relationship between the video block determining to disable use of a scaling matrix for the transformation; and performing the transformation using a scaling matrix, the scaling matrix being adapted to scale the image blocks during the transformation. The coefficients are used to scale at least some of the coefficients that represent the block.

[0019] In another exemplary aspect, another method of image processing is disclosed. The method includes: A transform skip mode is used for transforming between the coded representation of the image block and the video block. The applicability of the in-loop filter is determined by whether it is enabled for the above transformation. and performing this conversion based on the applicability of the in-loop filter. and, in the transform skip mode, during the transform, the video block is represented. In this case, the transform is skipped for at least some of the coefficients.

[0020] In another exemplary aspect, another method of image processing is disclosed. The method includes: For the conversion between the image block and the coded representation of this image block, a scaling matrix is ​​defined as The same scaling is used for transforms based on intra-block coding and intra-block copy coding. a scaling matrix is ​​selected; and performing a pre-calculation using the selected scaling matrix. and performing a scaling transformation on at least some of the image blocks, the scaling matrix being It is used to scale some factors.

[0021] In another exemplary aspect, another method of image processing is disclosed. The method includes: A transformation selected for transforming between an image block and a coded representation of the image. selecting a scaling matrix based on the matrix; and performing the transformation using a scaling matrix of the image block; is used to scale at least some of the coefficients, the transformation matrix being A selection of the image blocks is used to transform at least some of the coefficients of the image blocks during conversion. do.

[0022] In another exemplary aspect, another method of image processing is disclosed. The method includes: For the conversion between the image block and the coded representation of the image, a second image block is selected for the conversion. selecting a scaling matrix based on the linear transformation matrix; and performing the transformation using a scaling matrix, the scaling matrix being adapted to scale the image. at least some of the coefficients of the secondary clock are used to scale the secondary clock; A transformation matrix transforms at least some residual coefficients of the video block during the transformation. Used for:

[0023] In another exemplary aspect, another method of image processing is disclosed. For an image block having a shape of determining a scaling matrix used to convert between the encoded representation and the current A syntax element in the scaling matrix signals the scaling matrix. and performing the transformation based on a scaling matrix, the scaling matrix being is used to scale at least some coefficients of the image block to

[0024] In another exemplary aspect, another method of video processing is disclosed. By partially applying the scaling matrix when converting between the coded representation of the block and the video block, determining a first set of positions for the video block; and determining whether the first set of positions corresponds to the first set of positions for the video block. The scaling matrix is ​​applied to the remaining positions, and the scaling matrix is ​​nullified. and performing the transformation by locally applying a ring matrix.

[0025] In another exemplary aspect, another method of video processing is disclosed. A scaling matrix is ​​applied when converting between the encoded representation of the block and the video block. To judge, performing the transformation based on the scaling matrix, The encoded representation signals the number of elements of the scaling matrix, the number being indicative of the transformation. The reliance on the application of a zeroing coefficient in

[0026] In another exemplary aspect, another method of image processing is disclosed. The method includes: When converting between image blocks and a coded representation of the image, a sub-segment of the image block of a particular type is generated. determining a single quantization matrix to be used based on the size of the image; and performing said conversion using said

[0027] In yet another exemplary embodiment, the method is provided in the form of processor executable code. The method may be embodied in and stored on a computer readable program medium.

[0028] In yet another exemplary embodiment, a device configured or operable to perform the method described above. A possible device is disclosed, which is programmed to implement the method. A processing unit may be included.

[0029] In yet another exemplary aspect, a video decoder device as described herein The method may be implemented.

[0030] These and other aspects and features of the disclosed technology are set forth in the drawings, description and claims. This is explained in more detail in [Brief description of the drawings]

[0031] [Figure 1] FIG. 1 is a block diagram of an example implementation of a video encoder. [Diagram 2] FIG. 2 shows an example of a secondary transformation. [Diagram 3] FIG. 3 illustrates the Reduced Secondary Transform (RST). [Figure 4] FIG. 4 is a diagram illustrating two scalar quantization modules used in the proposed dependent quantization method. [Diagram 5]FIG. 5 illustrates the state transitions and quantization module selection for the proposed dependent quantization. [Figure 6A] FIG. 6A shows an example of a diagonal scan order. [Figure 6B] FIG. 6B shows an example of a diagonal scan order. [Figure 7] FIG. 7 shows an example of positions selected for QM signaling (32×32 transform size). [Figure 8] FIG. 8 shows an example of positions selected for QM signaling (64×64 transform size). [Figure 9] FIG. 9 shows an example of applying zeroing to coefficients. [Figure 10] FIG. 10 shows an example where only selected elements in a dashed region (eg, an M×N region) are signaled. [Figure 11] FIG. 11 is a block diagram illustrating an example of a video processing hardware platform. [Figure 12] FIG. 12 is a flow chart illustrating an example of a video processing method. [Figure 13] FIG. 13 is a block diagram illustrating a video decoder. [Figure 14] FIG. 14 is a block diagram illustrating an example video processing system in which various techniques disclosed herein may be implemented. [Figure 15] FIG. 15 is a block diagram illustrating an example video encoding system that can utilize techniques of this disclosure. [Figure 16] FIG. 16 is a block diagram illustrating a video encoder. [Figure 17] FIG. 17 is a flow chart illustrating an example of a video processing method. [Figure 18] FIG. 18 is a flow chart illustrating an example of a video processing method. [Figure 19] FIG. 19 is a flow chart illustrating an example of a video processing method. [Figure 20] FIG. 20 is a flow chart illustrating an example of a video processing method. [Figure 21]FIG. 21 is a flow chart illustrating an example of a video processing method. [Figure 22] FIG. 22 is a flow chart illustrating an example of a video processing method. [Figure 23] FIG. 23 is a flow chart illustrating an example of a video processing method. [Figure 24] FIG. 24 is a flow chart illustrating an example of a video processing method. [Diagram 25] FIG. 25 is a flow chart illustrating an example of a video processing method. [Figure 26] FIG. 26 is a flow chart illustrating an example of a video processing method. [Figure 27] FIG. 27 is a flow chart illustrating an example of a video processing method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] The disclosed technology embodiment is a method for improving compression performance by using existing video coding standards ( For example, HEVC, H.265) and future standards. Chapter headings are used to improve the readability of the description, and the description or embodiments (and This document does not limit the scope of the specification (and / or implementation) to only the respective chapters.

[0033] 1. Summary of the Invention This specification relates to an image / video coding technology. This relates to the quantization matrix. It may be applied to existing video coding standards such as HEVC. and applied to finalize the standard (Versatile Video Coding). The present invention is also applicable to future video encoding standards or video codecs.

[0034] 2.Background technology Video coding standards have emerged primarily through the development of well-known ITU-T and ISO / IEC standards. ITU-T has developed H.261 and H.263, and ISO / IEC has developed MPEG- 1 and MPEG-4 Visual, and the two organizations have jointly developed H.262 / MPEG-2 Video o and H.264 / MPEG-4 AVC(Advanced Video Coding ) and co-created the H.265 / HEVC standard. Since H.262, video coding standards have It is based on a hybrid video coding structure that utilizes inter prediction and transform coding. In 2015, VCEG and MPEG jointly established the VCEG-MPEG 2.0 standard for video coding. We established JVET (Joint Video Exploration Team). Since then, many new methods have been adopted by JVET and JEM (Joint Exp It has been incorporated into the reference software called the Loration Mode. In April 2018, VCEG (Q6 / 16) and ISO / IEC JTC1 SC29 / WG1 1 (MPEG) and the Joint Video Expert Team (JVET) The VVC standard was established with the goal of reducing the bit rate by 50% compared to HEVC. They are teaming up.

[0035] The latest version of VVC draft, namely Versatile Video Codi ng (draft 5) can be found at:

[0036] http: / / phenix.it-sudparis.eu / jvet / doc_e nd_user / current_document.php?id=6640

[0037] The latest reference software for VVC, called VTM, can be found at:

[0038] https: / / vcgit.hhi.fraunhofer.de / jvet / VVC Software_VTM / tags / VTM-4.0

[0039] 2.1. Color Space and Chroma Subsampling A color space, also known as a color model (or color system), is a set of numbers that represent a range of colors. It is an abstract mathematical model that is easily described as a tuple, typically consisting of 3 or Four values ​​or color components (e.g. RGB). Basically, a color space is a coordinate system and a subspace. It is a synthesis of the above.

[0040] For video compression, the most frequently used color spaces are YCbCr and RGB.

[0041] YCbCr, Y'CbCr, or Y Pb / Cb Pr / Cr is YCBCR or Also known as Y'CBCR, it is used in color imaging pipelines in video and digital photography systems. A family of color spaces that are used as part of the RGB color space. Y' is the luminance component, and CB and C R is the blue-difference and red-difference chroma components. Y' (which has a prime number) is distinct from Y, and Y is the luminance, and the light intensity is nonlinearly coded based on the gamma-corrected RGB primaries. This means:

[0042] Chroma subsampling occurs because the human visual system has a lower perception of color difference than it does of luminance. Taking advantage of this, the chrominance information is implemented with a lower resolution than the luminance information to produce an image. This is a method of encoding the above.

[0043] 2.1.1. 4:4:4 Each of the three Y'CbCr components has the same sample rate and is therefore This method is used by high-end film scanners and cinema post-production equipment. It can be used in production.

[0044] 2.1.2. 4:2:2 The two chrominance components are sampled at half the luma sample rate, resulting in a horizontal chrominance resolution of The image resolution is halved. This results in a nearly identical image quality to the uncompressed video signal with little or no visible difference. Reduce the signal bandwidth by a factor of three.

[0045] 2.1.3. 4:2:0 In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but this scheme Since the Cb and Cr channels are sampled only on every other line, the vertical resolution is The image resolution is halved. Therefore, the data rate remains the same. Cb and Cr are the horizontal The image is subsampled by a factor of 2 in both the horizontal and vertical directions. There are three variants of the 4:2:0 scheme: In MPEG-2, Cb and Cr are horizontally aligned. They are located between pixels in the perpendicular direction (located between the grid lines). ● JPEG / JFIF, H.261, and MPEG-1, Cb, and Cr is located between the grids midway between alternating luminance samples. ● In 4:2:0 DV, Cb and Cr are horizontally co-located. Vertically In the above, they co-locate alternately.

[0046] 2.2. Typical video codec encoding flow Figure 1 shows three in-loop filtering blocks: the deblocking filter ( VVC encoder blocks including DF, Sample Adaptive Offset (SAO) and ALF An example of a block diagram is shown. Unlike DF (which uses predefined filters), SAO and ALF and ALF use the original samples of the current picture and add offsets to them, respectively. and by applying a Finite Impulse Response (FIR) filter. The original samples are then reconstructed using the coding side information that signals the offset and filter coefficients. The ALF reduces the mean square error between the input and the reconstructed samples. It sits at one stage and attempts to capture and correct artifacts produced by the previous stage. It can be seen as a tool to

[0047] 2.3.Quantization Unit The well-known spatial frequency sensitivity of the human visual system (HVS) is Supports modern image and video formats, including .264 / AVC High Profile, and HEVC and is the key driver behind many aspects of the design of video coding algorithms and standards. do.

[0048] The quantization matrix used in MPEG2 is an 8x8 matrix. The block sizes of the quantization matrices include both 4×4 and 8×8. These QMs are , SPS (Sequence Parameter Set) and PPS (Picture Parameter Set) The compression method for QM signaling in H.264 / AVC is differential pulse coding. It is a digital pulse modulation (DPCM).

[0049] H.264 / AVC High Profile supports 4x4 block size and A block size of 8×8 is used. Six QMs for a 4×4 block size (i.e. i.e. separate matrices for intra / inter coding and Y / Cb / Cr components) and Two QMs for 8x8 block size (i.e. for intra / inter Y components) For all quantization matrices (10 separate matrices), only 8 quantization matrices need to be coded into the bitstream. do.

[0050] 2.4. Transform and Quantization Design in VVC 2.4.1. Conversion HEVC uses a finite precision approximation of the Discrete Cosine Transform (DCT), from 4x4 to 32x32 HEVC also specifies 4x4 luma intra prediction residuals. An alternative 4x4 integer transform based on the discrete sine transform (DST) for use in the difference block. In addition, for certain block sizes, conversion skipping may be permitted. .

[0051] nS=4,8,16,32,DCT-II transformation matrix cij(i,j=0.nS-1) is defined as follows: nS=4 {64,64,64,64} {83,36,-36,-83} {64,-64,-64,64} {36,-83,83,-36} nS=8 {64,64,64,64,64,64,64,64,64} {89,75,50,18,-18,-50,-75,-89} {83,36,-36,-83,-83,-36,36,83} {75,-18,-89,-50,50,89,18,-75} {64,-64,-64,64,64,-64,-64,64} {50,-89,18,75,-75,-18,89,-50} {36,-83,83,-36,-36,83,-83,36} {18,-50,75,-89,89,-75,50,-18} nS=16 {64,64,64,64,64,64,64,64,64,64,64,64,64, 64,64,64} {90,87,80,70,57,43,25,9,-9-25-43-57-70-8 0-87-90} {89,75,50,18-18-50-75-89-89-75-50-18,18, 50,75,89} {87,57,9-43-80-90-70-25,25,70,90,80,43,- 9-57-87} {83,36-36-83-83-36,36,83,83,36-36-83-83- 36,36,83} {80,9-70-87-25,57,90,43-43-90-57,25,87,7 0,-9-80} {75-18-89-50,50,89,18-75-75,18,89,50-50- 89-18,75} {70-43-87,9,90,25-80-57,57,80-25-90,-9,8 7,43-70} {64-64-64,64,64-64-64,64,64-64-64,64,64- 64-64,64} {57-80-25,90,-9-87,43,70-70-43,87,9-90,2 5,80-57} {50-89,18,75-75-18,89-50-50,89-18-75,75, 18-89,50} {43-90,57,25-87,70,9-80,80,-9-70,87-25-5 7,90-43} {36-83,83-36-36,83-83,36,36-83,83-36-36, 83-83,36} {25-70,90-80,43,9-57,87-87,57,-9-43,80-9 0,70-25} {18-50,75-89,89-75,50-18-18,50-75,89-89, 75-50,18} {9-25,43-57,70-80,87-90,90-87,80-70,57-4 3,25,-9} nS=32 {64,64,64,64,64,64,64,64,64,64,64,64,64, 64,64,64,64,64,64,64,64,64,64,64,64,64,6 4,64,64,64,64,64} {90,90,88,85,82,78,73,67,61,54,46,38,31, 22,13,4,-4-13-22-31-38-46-54-61-67-73-78 -82-85-88-90-90} {90,87,80,70,57,43,25,9,-9-25-43-57-70-8 0-87-90-90-87-80-70-57-43-25,-9,9,25,43, 57,70,80,87,90} {90,82,67,46,22,-4-31-54-73-85-90-88-78- 61-38-13,13,38,61,78,88,90,85,73,54,31,4 -22-46-67-82-90} {89,75,50,18-18-50-75-89-89-75-50-18,18, 50,75,89,89,75,50,18-18-50-75-89-89-75-5 0-18,18,50,75,89} {88,67,31-13-54-82-90-78-46,-4,38,73,90, 85,61,22-22-61-85-90-73-38,4,46,78,90,82 ,54,13-31-67-88} {87,57,9-43-80-90-70-25,25,70,90,80,43,- 9-57-87-87-57,-9,43,80,90,70,25-25-70-90 -80-43,9,57,87} {85,46-13-67-90-73-22,38,82,88,54,-4-61- 90-78-31,31,78,90,61,4-54-88-82-38,22,73 ,90,67,13-46-85} {83,36-36-83-83-36,36,83,83,36-36-83-83- 36,36,83,83,36-36-83-83-36,36,83,83,36-3 6-83-83-36,36,83} {82,22-54-90-61,13,78,85,31-46-90-67,4,7 3,88,38-38-88-73,-4,67,90,46-31-85-78-13 ,61,90,54-22-82} {80,9-70-87-25,57,90,43-43-90-57,25,87,7 0,-9-80-80,-9,70,87,25-57-90-43,43,90,57 -25-87-70,9,80} {78,-4-82-73,13,85,67-22-88-61,31,90,54- 38-90-46,46,90,38-54-90-31,61,88,22-67-8 5-13,73,82,4-78} {75-18-89-50,50,89,18-75-75,18,89,50-50- 89-18,75,75-18-89-50,50,89,18-75-75,18,8 9,50-50-89-18,75} {73-31-90-22,78,67-38-90-13,82,61-46-88, -4,85,54-54-85,4,88,46-61-82,13,90,38-67 -78,22,90,31-73} {70-43-87,9,90,25-80-57,57,80-25-90,-9,8 7,43-70-70,43,87,-9-90-25,80,57-57-80,25 ,90,9-87-43,70} {67-54-78,38,85-22-90,4,90,13-88-31,82,4 6-73-61,61,73-46-82,31,88-13-90,-4,90,22 -85-38,78,54-67} {64-64-64,64,64-64-64,64,64-64-64,64,64- 64-64,64,64-64-64,64,64-64-64,64,64-64-6 4,64,64-64-64,64} {61-73-46,82,31-88-13,90,-4-90,22,85-38- 78,54,67-67-54,78,38-85-22,90,4-90,13,88 -31-82,46,73-61} {57-80-25,90,-9-87,43,70-70-43,87,9-90,2 5,80-57-57,80,25-90,9,87-43-70,70,43-87, -9,90-25-80,57} {54-85,-4,88-46-61,82,13-90,38,67-78-22, 90-31-73,73,31-90,22,78-67-38,90-13-82,6 1,46-88,4,85-54} {50-89,18,75-75-18,89-50-50,89-18-75,75, 18-89,50,50-89,18,75-75-18,89-50-50,89-1 8-75,75,18-89,50} {46-90,38,54-90,31,61-88,22,67-85,13,73- 82,4,78-78,-4,82-73-13,85-67-22,88-61-31 ,90-54-38,90-46} {43-90,57,25-87,70,9-80,80,-9-70,87-25-5 7,90-43-43,90-57-25,87-70,-9,80-80,9,70- 87,25,57-90,43} {38-88,73,-4-67,90-46-31,85-78,13,61-90, 54,22-82,82-22-54,90-61-13,78-85,31,46-9 0,67,4-73,88-38} {36-83,83-36-36,83-83,36,36-83,83-36-36, 83-83,36,36-83,83-36-36,83-83,36,36-83,8 3-36-36,83-83,36} {31-78,90-61,4,54-88,82-38-22,73-90,67-1 3-46,85-85,46,13-67,90-73,22,38-82,88-54 ,-4,61-90,78-31} {25-70,90-80,43,9-57,87-87,57,-9-43,80-9 0,70-25-25,70-90,80-43,-9,57-87,87-57,9, 43-80,90-70,25} {22-61, 85-90, 73-38, -4, 46-78, 90-82, 54-13- 31,67-88,88-67,31,13-54,82-90,78-46,4,38 -73,90-85,61-22} {18-50,75-89,89-75,50-18-18,50-75,89-89, 75-50,18,18-50,75-89,89-75,50-18-18,50-7 5,89-89,75-50,18} {13-38, 61-78, 88-90, 85-73, 54-31, 4, 22-46, 6 7-82, 90-90, 82-67, 46-22, -4, 31-54, 73-85, 90 -88,78-61,38-13} {9-25, 43-57, 70-80, 87-90, 90-87, 80-70, 57-4 3,25,-9,-9,25-43,57-70,80-87,90-90,87-80 ,70-57,43-25,9} {4-13, 22-31, 38-46, 54-61, 67-73, 78-82, 85-8 8,90-90,90-90,88-85,82-78,73-67,61-54,46 -38,31-22,13,-4}

[0052] 2.4.2.Quantization The HEVC quantizer design is similar to that of H.264 / AVC, where 0~ Quantization parameters (QP) ranging from 51 (for 8-bit video sequences) to QP values Every time an increase of is made by 6, it maps to a quantization step size that doubles. However, The key difference is that in HEVC, the descaling matrix of H.264 / AVC is The transformation base norm correction factor included in the quantizer is no longer needed, simplifying the design of the quantizer. For rate control and perceptual quantization, a small quantization group of 8x8 samples is used. You can send a QP value (in the form of a delta QP) to the group. Calculate the delta QP The QP prediction module for this uses a combination of the left, top and previous QP values. C also uses quantization matrices for all transform block sizes, Supports wavenumber-dependent quantization, as described in detail in Section 2.4.3.

[0053] Quantized transform coefficient q ij (i,j=0..nS-1) is the conversion coefficient d ij It is derived from (i,j=0..nS-1) as follows: q ij =(d ij *f[QP%6]+offset)>>(29+QP / 6-nS-BitDepth),with i,j=0,...,nS-1 where f[x]={26214,23302,20560,18396,16384,14564},x=0,…,5 2 28+QP / 6-nS-BitDepth <offset<2 29+QP / 6-nS-BitDepth

[0054] QP represents the quantization parameter for one transform unit, and BitDepth is currently Represents the bit depth associated with the color components of

[0055] In HEVC, the range of QP is [0,51].

[0056] 2.4.3. Quantization Unit Quantization matrices (QMs) are used in image coding standards such as JPEG and JPEG-2000, and in M It is used in video standards such as PEG2, MPEG4, and H.264 / AVC. The subjective quality can be improved by frequency weighting the different frequency coefficients. In the HEVC standard, the size of the quantization block can be increased up to 32x32. In the bitstream, Q of sizes 4x4, 8x8, 16x16, and 32x32 are supported. For each block size, the intra / inter prediction type and The Y / Cb / Cr color components require different quantization matrices. Four block sizes: 4x4, 8x8, 16x16, and 32x32. In this case, two matrices (inter and separate matrices for Y,U,V components) should be coded.

[0057] The parameters for the quantization matrix may be copied directly from the reference quantization matrix, or may be explicitly signaled. In the case where it is explicitly signaled, the first parameter (the matrix (0,0) component value) is directly encoded. Then, predictive encoding by raster scanning the matrix The remaining parameters are encoded using the

[0058] The coding and signaling of scaling matrices in HEVC supports three modes: Meanings: OFF, DEFAULT, and USER_DEFINED. For larger transform unit sizes (i.e. 16x16, 32x32), the scaling rows are The columns are upsampled from an 8x8 scaling matrix to a larger size. For scaling matrices with TB larger than 8x8, An additional DC value must be signaled.

[0059] The maximum number of encoded values ​​for one scaling matrix in HEVC is 64.

[0060] The DC value in DEFAULT mode is 16 for all TB sizes.

[0061] 2.4.3.1. Syntax and Semantics 7.3.2.2 Sequence Parameter Set RBSP Syntax 7.3.2.2.1 General sequence parameter set RBSP syntax

[0062] [Table 1]

[0063] 7.3.2.3 Picture parameter set RBSP syntax 7.3.2.3.1 General picture parameter set RBSP syntax

[0064] [Table 2]

[0065] 7.3.4 Scaling List Data Syntax

[0066] [Table 3]

[0067] If scaling_list_enabled_flag is 1, the scaling of the conversion coefficients is Specifies that a scaling list is used for ring processing. If _enabled_flag is 0, scaling is disabled for the scaling process of the conversion coefficients. The SEC specifies that the SEC will not use a filtering list. If sps_scaling_list_data_present_flag is 1 , specifies that the scaling_list_data() syntax structure is present in the SPS. If sps_scaling_list_data_present_flag is 0 stipulates that the scaling_list_data() syntax construct does not exist in the SPS. If it does not exist, sps_scaling_list_data_present_ The value of flag is inferred to be 0. pps_scaling_list_data_present_flag is 1 In this case, the scaling list specified by the active SPS and the scaling list specified by the PPS are The scaling list used for pictures that reference the PPS based on the ring list and Specifies that the data is derived. If esent_flag is 0, the scaling used for pictures that refer to the PPS. The scaling list data is equal to the scaling list data specified in the active SPS. When scaling_list_enabled_flag is 0, The value of pps_scaling_list_data_present_flag is 0. When scaling_list_enabled_flag is 1, sps_s caling_list_data_present_flag is 0 and pps_s When caling_list_data_present_flag is 0, the default The default scaling list data is the scaling list data specified in Section 7.4.5. This is used to derive the array ScalingFactor as described in the data semantics.

[0068] 7.4.5 Scaling List Data Semantics scaling_list_pred_mode_flag[sizeId][matr If [ixId] is 0, the values ​​of the scaling list are the same as those of the reference scaling list. The reference scaling list is specified as scaling_list_pr. ed_matrix_id_delta[sizeId][matrixId] It is determined. scaling_list_pred_mode_flag[sizeId][matr If [ixId] is 1, it is stipulated that the values ​​in the scaling list are explicitly signaled. Determine. scaling_list_pred_matrix_id_delta[sizeId ][matrixId] is ScalingList[sizeId][matrixI The reference scaling list for deriving [d] is defined as follows: - scaling_list_pred_matrix_id_delta[size If [matrixId] is 0, then i=0..Min(63,(1<<(4+ As specified in Table 7-5 and Table 7-6 for (sizeId<<1)))-1), Default scaling list ScalingList[sizeId][matri xId][i]. - Otherwise, the scaling list is taken from the reference scaling list as follows: It is inferred. refMatrixId=matrixId- scaling_list_pred_matrix_id_delta[size Id][matrixId]*(sizeId==3?3:1) (7-42) ScalingList[sizeId][matrixId][i]=Scalin gList[sizeId][refMatrixId][i] with i=0..Min(63,(1<<(4+(sizeId<<1)))- 1) (7-43) scaling_list_pred_matrix if sizeId is less than or equal to 2 The value of _id_delta[sizeId][matrixId] ranges from 0 to matrix If not (sizeId is 3), sc aling_list_pred_matrix_id_delta[sizeId][ The value of [matrixId] should be in the range from 0 to matrixId / 3. . Table 7-3 - sizeId Specification

[0069] [Table 4]

[0070] Table 7-4 - Specifications of matrixId based on sizeId, prediction mode, and color components Mr.

[0071] [Table 5]

[0072] scaling_list_dc_coef_minus8[sizeId-2][m atrixId]+8 If sizeId is 2, scaling to 16x16 size Value of the list variable ScalingFactor[2][matrixId][0][0] If sizeId is 3, the Sc of the scaling list of 32x32 size is specified. Specify the value of alignmentFactor[3][matrixId][0][0]. aling_list_dc_coef_minus8[sizeId-2][matr The value of [ixId] should be in the range of -7 to 247. scaling_list_pred_mode_flag[sizeId][matr ixId] is 0, scaling_list_pred_matrix_id_delt If a[sizeId][matrixId] is 0 and sizeId is greater than 1, then sc aling_list_dc_coef_minus8[sizeId-2][matr The value of [ixId] is inferred to be 8. scaling_list_pred_matrix_id_delta[sizeId If ][matrixId] is not 0 and sizeId is greater than 1, scaling _list_dc_coef_minus8[sizeId-2][matrixId] The value of scaling_list_dc_coef_minus8[sizeId-2 ][refMatrixId], where refMatrixId The value is given by equation 7-42. scaling_list_delta_coef is scaling_list_p If red_mode_flag[sizeId][matrixId][i] is 1, If so, the current matrix coefficient ScalingList[sizeId][matrixId][i ] and the previous matrix coefficient ScalingList[sizeId][matrixId][i The value of scaling_list_delta_coef specifies the difference between - The value must be between 128 and 127. ScalingList[sizeId][matrix The value of [Id] [i] is greater than 0. Table 7-5 - ScalingList[0][matrixI for i=0..15 d][i] default value specification

[0073] [Table 6]

[0074] Table 7-6 - ScalingList[1..3][matr for i=0..63 Default value specification for [ixId][i]

[0075] [Table 7]

[0076] 4-dimensional array ScalingFactor[sizeId][matrixId][x] [y], with x, y=0..(1<<(2+sizeId))-1 is the The sizeId is multiplied according to the specified variable sizeId and the matrixId specified in Table 7-4. Specifies the rate array. ScalingFactor[0][matr ixId][][] is derived as follows: ScalingFactor[0][matrixId][x][y]=Scali ngList[0][matrixId][i] (7-44) with i=0..15,matrixId=0..5,x=ScanOrde r[2][0][i][0],and y=ScanOrder[2][0][i][1 ] ScalingFactor[1][matr ixId][][] is derived as follows: ScalingFactor[1][matrixId][x][y]=Scali ngList[1][matrixId][i] (7-45) with i=0..63,matrixId=0..5,x=ScanOrde r[3][0][i][0],and y=ScanOrder[3][0][i][1 ] ScalingFactor[2][ma The trixId][][] is derived as follows: ScalingFactor[2][matrixId][x*2+k][y*2+ j]=ScalingList[2][matrixId][i] (7-46) with i=0..63,j=0..1,k=0..1,matrixId=0 ..5,x=ScanOrder[3][0][i][0],and y=ScanOr der[3][0][i][1] ScalingFactor[2][matrixId][0][0]=scali ng_list_dc_coef_minus8[0][matrixId]+8 (7-47) with matrixId=0..5 ScalingFactor[3][ma The trixId][][] is derived as follows: ScalingFactor[3][matrixId][x*4+k][y*4+ j]=ScalingList[3][matrixId][i] (7-48) with i=0..63,j=0..3,k=0..3,matrixId=0 ,3,x=ScanOrder[3][0][i][0],and y=ScanOrd er[3][0][i][1] ScalingFactor[3][matrixId][0][0]=scali ng_list_dc_coef_minus8[1][matrixId]+8 (7-49) with matrixId=0,3 If ChromaArrayType is 3, matrixId=1,2,4,5 , element ScalingFactor[3] of the chroma quantization matrix of size 32x32 [matrixId][][] is derived as follows: ScalingFactor[3][matrixId][x*4+k][y*4+ j]=ScalingList[2][matrixId][i] (7-50) with i=0..63,j=0..3,k=0..3,x=ScanOrde r[3][0][i][0],and y=ScanOrder[3][0][i][1 ] ScalingFactor[3][matrixId][0][0]=scali ng_list_dc_coef_minus8[0][matrixId]+8 (7-51)

[0077] 2.5. Transform and Quantization Design in VVC 2.5.1.MTS (Multiple Conversion Selection) The family of discrete sinusoidal transforms includes the well-known discrete Fourier, cosine, sine, and culler functions. Includes the Loewe (first-order Markov condition) transformation. Among all members, the cosine Eight types of transformations based on functions, and eight types of transformations based on sine functions, namely That is, DCT-I, II, ···, VIII and DST-I, II, ···, respectively. , VIII. These discrete cosine and sine transform variants are The different symmetries of the symmetric-periodic sequences are used in the proposed method. The selected types of DCT and DST transformation basis functions are summarized in Table 1 below. . Table 1. Transform-based functions of DCT-II / V / VIII, DST-I / VII for N-point input

[0078] [Table 8]

[0079] For one block, either transform skip or DCT2 / DST7 / DCT8 This method is called multiple transform selection. This is called MTS (Ansform Selection).

[0080] To control the MTS scheme, at the SPS level, intra and inter When MTS is enabled in SPS, , a CU level flag is signaled to indicate whether MTS is applied. Here, MTS applies only to luminance. The MTS CU level flags are A signal is generated when the condition is met. - Width and height are both less than 32 - CBF flag is 1

[0081] If the MTS CU flag is zero, then DCT2 is applied in both directions. However, If the MTS CU flag is 1, two other flags are additionally signaled, which are The mapping types are as shown in the table below. When converting matrix precision, the primary conversion code is 8 bits. Therefore, all transform cores used in HEVC use the 4-point DCT -2, DST-7, 8 point, 16 point, 32 point DCT-2, etc. Also, 64-point DCT-2, 4-point DCT-8, 8-point, 16-point Other conversion cores such as 32-point DST-7 and DCT-8 use 8-bit primary Use the retransformation core. Table 6. Transform-based functions of DCT-II / V / VIII, DST-I / VII for N-point input

[0082] [Table 9]

[0083] As in HEVC, the residual of a block may be coded in transform skip mode To avoid redundancy in syntax encoding, CU level MTS_CU_flag must be non-zero. If not, the transform skip flag is not signaled. is the same as for MTS in JEM4, where the block width and height are both 3 If less than or equal to 2, it indicates that transform skip is applicable to the CU.

[0084] 2.5.1.1. High Frequency Nulling In VTM4, large block size conversions up to 64x64 in size are enabled. This is mainly useful for high resolution footage, e.g. 1080p and 4K sequences. For a transformation block whose size (width or height, or both width and height) is 64, Zero out the high frequency transform coefficients and keep only the low frequency coefficients. For example, for an M × N transform block, In the case of block, if the block width is M and the block height is N, when M is 64, the left 32 columns Similarly, if N is 64, then only the top 32 rows of transform coefficients are retained. When using transform skip mode for large blocks, the values ​​are zeroed out. Use the entire block without

[0085] To reduce the complexity of the large DST-7 and DCT-8, for DST-7 and DCT-8 blocks with a size of 32 (width or height, or both width and height) For the 16x16 low frequency region, only the coefficients in the low frequency region are preserved. It will be held.

[0086] 2.5.2. Narrowing Secondary Transformation In JEM, the secondary transform consists of the forward primary transform and quantization (at the encoder). applied between inverse quantization (at the decoder side) and between inverse primary transform (at the decoder side). As shown in Figure 2, depending on the block size, 4 × 4 (or 8 × 8) secondary For example, a 4x4 secondary transform is performed on small blocks (i.e., min (width, height) < 8), and the 8x8 secondary transform is applied to each 8x8 block. Applies to large blocks (i.e., min(width,height)>4).

[0087] Since a non-separable transformation is applied to the secondary transformation, this is a non-separable secondary transformation. Separable Secondary Transform (NSST) There are 35 transformation sets in total, with 3 non-separable transformation rows per transformation set. columns (kernels, each with a 16x16 matrix) are used.

[0088] According to the intra prediction direction, JVET-K0099 uses the shrinking secondary transform (RST) was introduced, and JVET-L0133 included 4 transformation sets (instead of 35 transformation sets). ) mapping is introduced. In this contribution, 8×8 and 4×4 blocks are For, respectively, 16×48 and 16×16 matrices are used. For convenience of notation, Let us express the x48 conversion as RST8x8 and the 16x16 conversion as RST4x4. Such methods have been adopted by the VVC in recent years.

[0089] FIG. 3 illustrates the Reduced Secondary Transform (RST).

[0090] The secondary forward and reverse transformations are separate processing steps from the primary transformation. This is the logical step.

[0091] For the encoder, we first do the primary forward transform, then the secondary forward transform and Decoder, CABAC bit decoding, And for inverse quantization, first perform the secondary inverse transform and then the primary inverse transform.

[0092] RST only applies to intra-coded TUs.

[0093] 2.5.3.Quantization In VTM4, the maximum QP is increased from 51 to 63, and the first QP signaling is changed accordingly. When encoding non-zero values ​​of slice_qp_delta, S The initial value of liceQpY is changed in the slice segment layer. Specifically, init_ The value of qp_minus26 has been modified to the range of -(26+QpBdOffsetY)~+37. Correct.

[0094] Also, the same HEVC scalar quantization is combined with a new concept called dependent scalar quantization. Dependent scalar quantization is used when the set of allowable reconstruction values ​​for a transform coefficient is , an approach that depends on the value of the transform coefficient level preceding the current transform coefficient level in the reconstruction order. The main advantage of this approach is that it does not require traditional independent scalars, such as those used in HEVC. Compared with the linear quantization, the allowable reconstruction vectors are more densely packed in the N-dimensional vector space. (N represents the number of transform coefficients in the transform block). For a given average number of allowable reconstructed vectors per original unit volume, the input vector and This means that the average distortion with respect to the closest reconstruction vector is reduced. The approach is as follows: (a) Two scalar quantization modules with different reconstruction levels are used. (b) defining the process of switching between two scalar quantization modules. This is achieved by setting

[0095] Figure 4 shows the two scalar quantization modules used in the proposed dependent quantization method. This is a diagram.

[0096] The two scalar quantization modules used, denoted Q0 and Q1, are shown in Figure 4. The positions of the available reconstruction levels are uniquely defined by the quantization step size Δ The scalar quantization module used (Q0 or Q1) is Instead, the quantization module used for the current transform coefficients is not explicitly signaled. The rule is determined by the parity of the transform coefficient level that precedes the current transform coefficient in the coding / reconstruction order. It is judged.

[0097] As shown in Figure 5, the switching between the two scalar quantization modules (Q0 and Q1) is , is realized by a state machine with four states, which can be expressed as four different values 0, 1, 2, 3. This is the first transform coefficient to precede the current transform coefficient in the coding / reconstruction order. The parity of the transform coefficient level that is applied to the transform block is uniquely determined. At the start, set the state equal to 0. The transform coefficients are sorted in scan order (i.e., as they are entropy-dependent). After the transform coefficients are reconstructed, they are reconstructed as shown in Figure 5. As shown, if the value of the transform coefficient level is k, it is updated as follows: 2.5.4. User-defined quantization matrices in JVET-N0847

[0098] In this contribution, we use default and user-defined scaling lines on top of VTM4.0. It is proposed to add support for signaling columns. This proposal is Larger size range (4x4 to 64x64 for luma, 2x2 to 32 for chroma) ×32), rectangular TB, dependent quantization, multiple transform selection (MTS), zeroing- out) Large transforms using high frequency coefficients (in the case of TB, one step of the scaling matrix (aligned to the block definition procedure), intra-subblock division (ISP), and intra-block The Intermediate Picture Code (IBC), also known as the Current Picture Reference CPR, is compliant with the Intermediate Picture Code (ICC).

[0099] Default and user-defined scaling on top of VTM4.0 to comply with It is proposed to add syntax to support signaling of the tagging matrix. - Three modes for scaling matrices: OFF, DEFAULT, and USER_DEF INED - Larger block size range (4x4 to 64x64 for luma, 2×2~32×32) - Rectangular Transformation Blocks (TBs) - Dependent quantization - Multiple conversion selection (MTS) - Large transforms with zeroed high frequency coefficients - Intra sub-block division (ISP) - Intra Block Copy (IBC, also known as Current Picture Reference CPR) shares the same QM as the intra-coded blocks. - The DEFAULT scaling matrix is ​​flat for all TB sizes, The default value is 16. - Scaling matrices shall not be applied for: ○ TS for all TB sizes ○ Secondary Transformation (aka RST)

[0100] 2.5.4.1. Signalling QM for squared transform size 2.5.4.1.1. Scanning Order of Elements in a Scaling Matrix These elements are coded in the same scan order used for coding the coefficients, i.e., diagonal scan order. An example of a diagonal scan order is shown in Figures 6A-6B.

[0101] 6A-6B show examples of diagonal scan orders. FIG. 6A shows an example of a scan direction. 6B shows the coordinates and scan order index of the individual elements.

[0102] The specifications corresponding to this order are defined as follows: 6.5.2 Upper Right Diagonal Scan Order Array Initialization Process This process requires the block width blkWidth and block size height blkHeigh t is entered.

[0103] The output of this process is the array diagScan[sPos][sComp]. The array index sPos ranges from 0 to (blkWidth*blkHeight). Specifies the scan position in the range from -1 to 0. If the minute is specified and the array index sComp is 1, the vertical component is specified. Based on the values ​​of blkWidth and blkHeight, the array diagScan is , which is derived as follows: i=0 x=0 y=0 stopLoop=FALSE while(!stopLoop) { while(y>=0) { if(x <blkWidth&&y<blkHeight){(6-11) diagScan[i][0]=x diagScan[i][1]=y i++ } y-- x++ } y=x x=0 if(i>=blkWidth*blkHeight) stopLoop=TRUE }

[0104] Encoding Selection Elements The DC value (i.e., the element at the top-left corner of the matrix with a scan index of 0) is Scaling matrices, coded separately for 16x16, 32x32 and 64x64 can be. For TB(N×N) of size 8×8 (N<=8) or less For TBs of size 8x8 or less, all elements in one scaling matrix are signal You will be notified. For TBs (N×N) larger than 8×8 (N>8)

[0105] If the size of the TB is greater than 8x8, then 64 in one 8x8 scaling matrix Only elements are signaled as the base scaling matrix. The 64 elements are m*X,m*Y), where m=N / 8 and X, Y are in [0...7]. In other words, one N×N block is divided into multiple m×m non-overlapping regions, and each region is Each of the two shares the same element, and this shared element is signaled.

[0106] To obtain a square matrix of size larger than 8×8, we use the 8×8 base scaling matrix as Upsample to the appropriate square size (i.e. 16x16, 32x32, 64x64) Pulling (by cloning the elements).

[0107] For example, 32x32 and 64x64 are used to indicate the selected position of the element to be signaled. Mark with circles. Each square represents one element.

[0108] FIG. 7 shows an example of positions selected for QM signaling (32×32 transform size).

[0109] FIG. 8 shows an example of positions selected for QM signaling (64×64 transform size). 2.5.4.2. Derivation of QM for non-square transformation sizes

[0110] There is no additional signaling in QM for non-square transform sizes. Instead, The QM for the size of the squared transform is derived from the QM for the size of the squared transform. An example is shown in Figure 7.

[0111] Specifically, when generating a scaling matrix for a rectangle TB, the following two cases are considered: think. 1. The height of a rectangular matrix H is greater than its width W. Then, S for a rectangle TB of size W×H A calingMatrix is ​​defined with size baseL×baseL as follows: baseL is min(log2(H),3).

[0112]

number

[0113] 2. The height of the rectangular matrix H is less than its width W. Then, the size of the matrix H is W×H. The scaling matrix is ​​calculated from the reference scaling matrix of size baseL×baseL as follows: where baseL is min(log2(W),3).

[0114]

number

[0115] Here, int(x) changes the value of x by truncating the fractional part.

[0116] Figure 8 shows examples of QM derivation of non-square blocks from square blocks: (a) 8 × 8 (b) the QM of a 2×8 block derived from that block; (c) the QM of a 2×8 block derived from that block This is an 8x2 block QM.

[0117] 2.5.4.3. QM Signaling for Zeroed Transformation Blocks Also, when applying zeroing of high frequency coefficients for 64-point transforms, the corresponding scaling The high frequencies of the TB matrix are also zeroed out. That is, if the width or height of the TB is 32 or more, As shown in Figure 9, we only keep the left or upper half of the coefficients and assign zeros to the remaining coefficients. Therefore, when obtaining a rectangular matrix according to equations (1) and (2), a check is performed to determine whether S Assign 0 to the corresponding element of calingMatrix(i,j). 2.5.4.4. Quantization matrix syntax and semantics

[0118] The SPS and PPS have the same syntax elements as HEVC. However, The signaling of the scaling list data syntax is changed as follows: 7.3.2.11 Scaling List Data Syntax

[0119] [Table 10]

[0120] 7.4.3.11 Scaling List Data Semantics scaling_list_pred_mode_flag[sizeId][mat If [rixId] is 0, the values ​​in the scaling list are the same as those in the reference scaling list. The reference scaling list is specified as scaling_list_p. By red_matrix_id_delta[sizeId][matrixId] It is prescribed. scaling_list_pred_mode_flag[sizeId][mat If [rixId] is 1, the values ​​in the scaling list are explicitly signaled. Prescribe. scaling_list_pred_matrix_id_delta[sizeI d][matrixId] is ScalingList[sizeId][matrix Id], and ScalingL The derivation of ist[sizeId][matrixId] is as follows: _list_pred_matrix_id_delta[sizeId][matri xId]. -scaling_list_pred_matrix_id_delta[size If [matrixId] is 0, the scaling list is Regarding (63,(1<<(sizeId<<1))-1), Table 7-15, Table 7-16, Table The default scaling list Scali, as specified in Table 7-17 and Table 7-18 Inferred from ngList[sizeId][matrixId][i]. - If not, the scaling list is taken from the reference scaling list as below: It is inferred. For sizeId=1…6, refMatrixId=matrixId-scaling_list_pred _matrix_id_delta[sizeId][matrixId]*(size Id==6?3:1) (7-XX) If sizeId is 1, the value of refMatrixId is not 0 or 3. If sizeId is less than or equal to 5, scaling_list_pred_mat The value of rix_id_delta[sizeId][matrixId] ranges from 0 to mat If not (sizeId is 6) , scaling_list_pred_matrix_id_delta[sizeI The value of [d] [matrixId] should be in the range from 0 to matrixId / 3. It is. Table 7-13 - sizeId Specification

[0121] [Table 11]

[0122] Table 7-14 - Specifications of matrixId based on sizeId, prediction mode and color components Mr.

[0123] [Table 12]

[0124] scaling_list_dc_coef_minus8[sizeId][mat rixId]+8If sizeId is 4, a 16x16 size scaling list Specify the value of the variable ScalingFactor[4][matrixId][0][0] and sizeId is 5, the Scali for 32x32 size scaling list Specify the value of ngFactor[5][matrixId][0][0] and sizeId ScalingFacto of the scaling list for 64x64 size when is 6 Specifies the value of r[6][matrixId][0][0]. scaling_list The value of _dc_coef_minus8[sizeId][matrixId] is -7~ It should be within the range of 247. scaling_list_pred_mode_flag[sizeId][mat rixId] is 0, scaling_list_pred_matrix_id_del If ta[sizeId][matrixId] is 0 and sizeId is greater than 3, then s caling_list_dc_coef_minus8[sizeId][matri The value of [xId] is inferred to be 8. scaling_list_pred_matrix_id_delta[sizeI If [d][matrixId] is not 0 and sizeId is greater than 3, then scalin g_list_dc_coef_minus8[sizeId][matrixId] The value is scaling_list_dc_coef_minus8[sizeId][r efMatrixId], where the value of refMatrixId is the formula Given by 7-XX. scaling_list_delta_coef is scaling_list_p If red_mode_flag[sizeId][matrixId][i] is 1, If so, the current matrix coefficient ScalingList[sizeId][matrixId][i ] and the previous matrix coefficient ScalingList[sizeId][matrixId][i The value of scaling_list_delta_coef specifies the difference between - The value must be between 128 and 127. ScalingList[sizeId][matrix The value of [Id][i] is greater than 0. If _mode_flag[sizeId][matrixId] is 1, and scal If ing_list_delta_coef does not exist, ScalingList[ The value of [sizeId][matrixId][i] is inferred to be 0. Table 7-15 - ScalingList[1][matrixId][i] defaults Specification of the value (i=0..3)

[0125] [Table 13]

[0126] Table 7-16 - ScalingList[2][matrix for i=0..15 Default value specification for [Id][i]

[0127] [Table 14]

[0128] Table 7-17 - Specification of default values of ScalingList[3...6][matrixId][i] for i = 0..63

[0129] [Table 15]

[0130] Table 7-18 - Specification of default values of ScalingList[6][matrixId][i] (i = 0..63)

[0131] [Table 16]

[0132] The 5D array ScalingFactor[sizeId][sizeId][matrixId][x][y] (x, y = 0..(1<<sizeId)-1) defines an array of ratios according to the variable sizeId defined in Table 7-13 and matrixId defined in Table 7-14. The elements ScalingFactor[1][matrixId][][] of the quantization matrix of size 2×2 are derived as follows. ScalingFactor[1][1][matrixId][x][y]=Sc alingList[1][matrixId][i] (7-XX) with i = 0..3, matrixId = 1, 2, 4, 5, x = DiagSc anOrder[1][1][i][0], and y = DiagScanOrder 1][1][i][1] ​​​​​​ScalingFactor[2][matr ixId][][] is derived as follows: ScalingFactor[2][2][matrixId][x][y]=Sc alingList[2][matrixId][i] (7-XX) with i=0..15,matrixId=0..5,x=DiagScan Order[2][2][i][0],and y=DiagScanOrder[2] [2][i][1] ScalingFactor[3][matr ixId][][] is derived as follows: ScalingFactor[3][3][matrixId][x][y]=Sc alingList[3][matrixId][i] (7-XX) with i=0..63,matrixId=0..5,x=DiagScan Order[3][3][i][0],and y=DiagScanOrder[3] [3][i][1] ScalingFactor[4][ma [trixId][] is derived as follows: ScalingFactor[4][4][matrixId][x*2+k][y *2+j]=ScalingList[4][matrixId][i](7-XX) with i=0..63,j=0..1,k=0..1,matrixId=0 ..5,x=DiagScanOrder[3][3][i][0],and y=Di agScanOrder[3][3][i][1] ScalingFactor[4][4][matrixId][0][0]=sc aling_list_dc_coef_minus8[0][matrixId]+8 (7-XX) with matrixId=0..5 Quantization matrix element ScalingFactor[5][matri xId][][] is derived as follows: ScalingFactor[5][5][matrixId][x*4+k][y *4+j]=ScalingList[5][matrixId][i] (7-XX) with i=0..63,j=0..3,k=0..3,matrixId=0 ..5,x=DiagScanOrder[3][3][i][0],and y=Di agScanOrder[3][3][i][1] ScalingFactor[5][5][matrixId][0][0]=sc aling_list_dc_coef_minus8[1][matrixId]+8 (7-XX) with matrixId=0..5 ScalingFactor[6][ma The trixId][][] is derived as follows: ScalingFactor[6][6][matrixId][x*8+k][y *8+j]=ScalingList[6][matrixId][i] (7-XX) with i=0..63,j=0..7,k=0..7,matrixId=0 ,3,x=DiagScanOrder[3][3][i][0],and y=Dia gScanOrder[3][3][i][1] ScalingFactor[6][6][matrixId][0][0 ]=scaling_list_dc_coef_minus8[2][matrixI d]+8 (7-XX) with matrixId=0,3 If ChromaArrayType is 3, matrixId=1,2,4,5 If so, the element ScalingFactor[ of the chroma quantization matrix of size 64x64 6][6][matrixId][][] is derived as follows: ScalingFactor[6][6][matrixId][x*8+k][y *8+j]=ScalingList[5][matrixId][i] (7-XX) with i=0..63,j=0..7,k=0..7,x=DiagScan Order[3][3][i][0],and y=DiagScanOrder[3] [3][i][1] ScalingFactor[6][6][matrixId][0][0]=sc aling_list_dc_coef_minus8[1][matrixId]+8 (7-XX) / / For non-square cases For a quantization matrix with a rectangular size, x=0..(1< <sizeIdW)-1,y=0..(1<<sizeIdH)-1 , 5-dimensional array ScalingFactor[ where sizeIdW != sizeIdH sizeIdW][sizeIdH][matrixId][x][y] are as shown in Table 7-19. The array of magnifications is specified according to the variables sizeIdW and sizeIdH specified in It is also derived as follows: ScalingList[sizeLid][matrixId][i] determines the size of the matrix. alingFactor[sizeIdW][sizeIdH][matrixId][ [x][y] can be generated. where sizeLId = max(sizeIdW, sizeIdH), sizeId W=0,1..6,sizeIdH=0,1..6,matrixId=0..5,x= 0..(1< <sizeIdW)-1,y=0..(1<<sizeIdH)-1,,x =DiagScanOrder[k][k][i][0], and y=DiagScan Order[k][k][i][1],k=min(sizeLId,3), and ratioW=(1< <sizeIdW) / (1<<k),ratioH=(1< <sizeIdH) / (1<<k),and ratioWH=(1<<abs(siz eIdW-sizeIdH), see rules below: - If (sizeIdW>sizeIdH), ScalingFactor[sizeIdW][sizeIdH][matrix Id][x][y] =ScalingList[sizeLId][matrixId][Raster 2Diag[(1< <k)*((y*ratioWH) / ratioW)+x / rati oW]] - others ScalingFactor[sizeIdW][sizeIdH][matrix Id][x][y] =ScalingList[sizeLId][matrixId][Raster 2Diag[(1< <k)*(y / ratioH)+(x*ratioWH) / rati oH]], Here, Raster2Diag[] is the raster scan position in one 8x8 block. is a function that transforms the vertex position to a diagonal scan position. / / If zeroing Quantization matrices with rectangular sizes are zeroed for samples that satisfy the following conditions: - x>32 - y>32 - The decoded tu is not encoded with the default transformation mode (1< <sizeId W)==32, x>16 - The decoded tu is not encoded with the default transformation mode (1< <sizeId H)==32, y>16 Table 7-19 - sizeIdW and sizeIdH specifications

[0133] [Table 17]

[0134] 2.6 Quantized Residual Block Differential Pulse Code Modulation In JVET-M0413, quantized residual block differential pulse code modulation (Q R-BDPCM) has been proposed to efficiently code screen content. .

[0135] The prediction directions used in QR-BDPCM are vertical and horizontal prediction modes. Intra prediction can be supported in the same way as intra prediction, depending on the prediction direction (horizontal or vertical prediction). The residual is quantized and the quantized residual is and its predictor (horizontal or vertical) quantized value. This is For a block of size M (rows) × N (columns), ri,j , 0≦i≦M-1, 0≦j≦N-1 are filled from the boundary samples of the top or left block. Using unmapped samples, the pixel to the left of the prediction block is vertically (for each line in the prediction block, copy the values ​​from the neighbor above Let Q(ri,j) be the prediction residual after intra prediction on the line (copying the line). ≦i≦M−1, 0≦j≦N−1 represents the quantized version of the residual r i,j, where The residual is the difference between the original block and the predicted block value. is applied to the quantized residual samples, resulting in a modified M× We obtain an array R of N. When the vertical BDPCM is signaled,

[0136]

number

[0137] For horizontal prediction, a similar rule applies: the residual quantized samples are calculated by the following formula: Obtained.

[0138]

number

[0139] The residual quantized samples r~i,j are sent to the decoder.

[0140] At the decoder side, the above calculation is reversed to obtain Q(ri,j), 0≦i≦M-1, 0≦j≦ For vertical prediction,

[0141]

number

[0142] For the horizontal direction,

[0143]

number

[0144] Add the dequantized residual Q-1(Q(ri,j)) to the intrablock prediction and Generates the configured sample values.

[0145] The main advantage of this scheme is that the reverse DPCM can be calculated on-the-fly during coefficient parsing. You can do this by simply adding the predictors during coefficient parsing, or by This can be done after the analysis.

[0146] The proposed changes to the QR-BDPCM text are shown below. 7.3.6.5 Coding Unit Syntax

[0147] [Table 18]

[0148] If dbpcm_flag[x0][y0] is 1, the luminance coding of the position (x0,y0) Specifies that the bdpcm_dir_flag is present in the coding unit that contains the block. do. bdpcm_dir_flag[x0][y0]=0 is used in the bdpcm block. Specifies that the prediction direction to be used is horizontal, otherwise it is vertical.

[0149] 3. Examples of embodiments and technical problems that the technology aims to solve In the current VVC design, the following problems exist from the viewpoint of quantization matrices. 1.For QR-BDPCM and BDPCM-blocks, no transformation is applied. Therefore, for such blocks, the scaling process is performed in the same way as for other transformation application blocks. Applying columns is not optimal. 2. IBC and intra coding modes share the same scaling matrix. But However, IBC is likely to be an inter-coding tool. Such a design is not reasonable. It seems likely. 3. Zeroing For large blocks, some elements are signaled but waste bits It is reset to zero during the decoding process. 4. In some frameworks, quantization is done by always applying the quantization matrix. Thus, the quantization matrix for small blocks is changed frequently. It's possible.

[0150] 4. List of embodiments and techniques The following detailed invention should be considered as examples to illustrate the general concept. These inventions should not be construed narrowly. Moreover, these inventions do not include any can be combined in a manner 1. The TS (Transform Skip) flag is signaled in the bitstream. If not, how to set the value of this flag depends on the encoding mode information. obtain. In one example, for a BDPCM / QR-BDPCM coded block, The group is inferred to be 1. b. In one example, for non-BDPCM / QR-BDPCM coded blocks, The lag is inferred to be 0. 2. Applying a scaling matrix to the BDPCM / QR-BDPCM encoded block It is proposed that this is possible. a. Alternatively, scaling for BDPCM / QR-BDPCM coded blocks It is proposed that processions not be allowed. b. As an example, the scaling matrix for BDPCM / QR-BDPCM coded blocks The method of selecting can be done in the same way as for the transform skip coding block. c. In one example, scaling for BDPCM / QR-BDPCM coding blocks The choice of the transformation matrix depends on whether one or more transformations are applied to the block. It can depend. i. In one example, one or more BDPCM / QR-BDPCM coding blocks When applying the above transformation, scaling matrices may be allowed. ii. In one example, the one or more transforms are BDPCM / QR-BDPCM codes. When applied to a coded block, the sampling rate for a BDPCM / QR-BDPCM coded block is The selection of the scaling matrix is ​​done in the same way as for one intra-coded block. This can be done. 3. In-loop filters (deblocking filters) and / or other post-reconstruction filters Whether to filter samples / edges between two adjacent blocks in the data. Which and / or how to filter depends on which of the two adjacent blocks It may depend on whether one or both are coded in transform skip (TS) mode. a. In-loop filters (deblocking filters) and / or other post-reconstruction filters Whether to filter samples / edges between two adjacent blocks in the filter The filter size and / or how to filter depends on whether two adjacent blocks are in T This may depend on whether the data is encoded in S, BDPCM, QR-BDPCM, or palette mode. do. b. In one example, the derivation of the boundary filtering strength is It may depend on one or both of the TS mode flags. c. In one example, for a sample located in a TS coding block, the deblocked frame Filters / Sample Adaptive Offset / Adaptive Loop Filters / Other Types of In-Loop Filters / Other post-reconstruction filters may be disabled. In one example, two adjacent blocks are both encoded in transform skip mode. If the two are filtered, there is no need to filter such edges between them. ii. In one example, one of the two adjacent blocks is in transform skip mode If one of them is coded and the other is not coded, the sample located in the TS coding block There is no need to filter pulls. d. Alternatively, a different filter ( For example, a smoother filter) may be allowed. e. In-loop filters (e.g., deblocking filters) and / or other reconstructions During post-processing filters (e.g. bilateral filters, diffusion filters), PCM / BDPCM / QR-BDPCM coded blocks and / or other types where no transformation is applied Such modes may be treated in the same way as those coded in TS mode, as described above. stomach. 4. Blocks coded in IBC and inter coding modes have the same scaling. Matrices may be shared. 5. The choice of scaling matrix may depend on the type of transformation matrix. In one example, the selection of the scaling matrix is ​​based on whether the block uses the default, such as DCT2. This may depend on whether or not a conversion is used. b. In one example, the scaling matrix is ​​calculated separately for multiple transformation matrix types. You may be notified. 6. The selection of the scaling matrix may depend on the motion information of one block. In one example, the selection of the scaling matrix is ​​determined based on whether the block is in sub-block coding mode. The degree of accuracy may depend on whether the image is encoded in affine mode (e.g., affine mode). b. In one example, scaling matrices are used separately for affine and non-affine modes. may be signaled. c. In one example, the selection of the scaling matrix is ​​performed when the block is an affine intra prediction model. The number of bits may depend on whether the bit is encoded in the 7. Instead of disabling the scaling matrix for the secondary transform coding block, It is proposed to use a scaling matrix. Let the size of the transformation block be K × L. To represent this, a secondary transformation is performed on the top left M × N block. a. Secondary transformation and / or reduced secondary transformation and / or rotation transformation For conversion, in tile group header / slice header / PPS / VPS / SPS The scaling matrix can be signaled. b. In one example, a scaling matrix may be selected. In one example, the elements of the scaling matrix for the top left M×N block are It may be signaled separately whether or not a candial transformation is applied. c. Alternatively, the scaling matrix is ​​applied only to regions where the secondary transformation is not applied. It may be possible. d. In one example, the rest of the image, except for the top left M×N region, is still scaled. A matrix may be applied. e. Alternatively, the scaling matrix can be applied only to the region where the secondary transformation is applied. It may be possible. 8. Alternative to deriving scaling matrices for non-square blocks from square blocks It is proposed to signal the scaling matrix for non-square blocks. In one example, the scaling matrix for a non-square block is , and the scaling matrix may be enabled. 9. Disable the use of scaling matrices for some positions and the rest of the block. It is proposed to validate a scaling matrix for the position. For example, if a block contains more than M*N positions, the top left M*N positions Only regions of may use the scaling matrix. b. For example, if a block contains more than M*N positions, the top M*N positions Only the position may use the scaling matrix. c. For example, if a block contains more than M*N positions, the left M*N positions Only positions in may use the scaling matrix. 10. The number of elements to be signaled in the scaling matrix depends on the zeroing applied. This may depend on whether In one example, for a 64x64 transform, only the top left MxN transform coefficients are kept. The remaining coefficients are all assumed to be zeroed out. Then, the number of elements to be signaled is may be derived as M / 8*N / 8. 11. For transformations with zeroing, the elements in the scaling matrix that lie in the zeroing region It is proposed to disable the signaling of . Assuming the case of K × L transformation, the upper left M × Only N transform coefficients are retained and all remaining coefficients are zeroed out. In one example, K=L=64 and M=N=32. b. In one example, in the scaling matrix corresponding to a location outside the upper left M × N region, In this case, the signaling of the element is omitted. FIG. 10 shows that only selected elements in the dashed region (e.g., M×N region) are signaled. Here is an example of what can be done. c. In one example, subsampling to select elements in a scaling matrix The ratio may be determined by K and / or L. For example, divide the transformation block into multiple sub-regions and set the size of each sub-region to Uw *Uh. Signal one element located within each sub-region of the upper left M × N region. This can be done. ii. Alternatively, the number of elements to be encoded may depend on M and / or N . 1) In one example, how many elements are coded for such a K×L transformation with zeroing? The zeroing that must be performed on the M×N transform block is different from that for an M×N transform block without zeroing. d. In one example, subsampling to select elements in a scaling matrix The ratio may be determined by M and / or N instead of K and L. i. For example, an M×N region is divided into multiple sub-regions. Each (M / Uw,N / U h) One element in a region may be signaled. ii. Alternatively, how many elements are there for such a K×L transformation with zeroing? should be coded the same as for an M×N transform block without zeroing. e. In one example, K=L=64, M=N=32, and Uw=Uh=8. 12. For a given block size, such as a small block, one quantization It is proposed to use matrices only. In one example, all blocks smaller than W x H are However, it may not be permitted to use more than one quantization matrix. b. In one example, all blocks with a width less than the threshold are split into two or more quanta. In some cases, it may not be permitted to use a quantization matrix. c. In one example, all blocks whose height is less than a threshold are quantized to two or more Sometimes it is not permitted to use matrices. d. In one example, the quantization matrix may not be applied to small size blocks. . 13. The black dots above do not apply any transformation (or no identity transformation) Other encoding methods are also applicable. In one example, the black dot above shows "TS / BDPCM / QR-BDPCM" as "P This applies to blocks coded in palette mode by replacing It is possible to use.

[0151] 5. Embodiments 5.1. Deblocking Filter Embodiment #1 [ka]

[0152] 8.8.2 Deblocking Filter Processing 8.8.2.1 General The input to this process is the reconstructed picture before deblocking, i.e., the array recPictur If eL and ChromaArrayType is not 0, the array recPict ureCb and recPictureCr. The output of this process is the modified reconstructed picture after deblocking, i.e., the array recPi ctureL and ChromaArrayType is not 0, the array recPi ctureCb and recPictureCr. First, we select the vertical edges of the picture, and then we correct them with the vertical edge filtering process. The horizontal edges of the picture are filtered using the sample obtained by the filter. Vertical and horizontal edges in the CTB are processed separately for each coding unit. The vertical edge of a coding block in a coding unit is Starting with The horizontal edges of the coding blocks in the coding unit are filtered to Starting from the upper edge of the coding block and moving towards the lower edge of the coding block, The edges are filtered in order as they proceed through the edges. NOTE – In this specification, the filtering process is described on a picture-by-picture basis. If the decoder properly considers the order of operations, the filter can generate the same output value. The tarring process may be performed on a coding unit basis with equivalent results. The deblocking filter process removes all coding of a picture, except for the following types of edges: It is applied to the edges of sub-blocks and to the edges of the transformed block. - edges on the picture border, - pps_loop_filter_across_virtual_boundar If ies_disabled_flag is set to 1, then the edges that fit the virtual borders of the picture are J - loop_filter_across_bricks_enabled_flag If is 0, the edges that meet the brick boundaries - slice_loop_filter_across_slices_enable d_flag is 0 or slice_deblocking_filter_d If isabled_flag is 1, the edge that matches the top or left border of the slice J - slice_deblocking_filter_disabled_flag Edges in the slice that are 1 - Edges that do not correspond to the 8x8 sample grid boundaries of the component being considered - Edges in chroma components where both sides of the edge use inter prediction - An edge of a chroma transform block that is not an edge of the associated transform unit - IntraSubPartitionsSplit value is ISP_NO_SPLIT The edges of the entire luminance transformation block of the coding unit are not [ka]

[0153] 5.2. Scaling Matrix Implementation #2 This chapter provides an example of sunspot 11.d from Chapter 4.

[0154] [ka] ScalingFactor[6][ma The trixId][][] is derived as follows: [ka] Also, [ka] ScalingFactor[6][6][matrixId][0][0]=sc aling_list_dc_coef_minus8[2][matrixId]+8 (7-XX) with matrixId=0,3 / / If zeroing Quantization matrices with rectangular sizes are zeroed for samples that satisfy the following conditions: [ka] - The decoded tu is not encoded by the default transformation mode (1< <size IdW)==32, x>16 - The decoded tu is not encoded in the default transformation mode (1< <sizeI dH)==32, y>16

[0155] 5.3. Scaling Matrix Implementation #3 This chapter provides examples of black points 9 and 11.c from Chapter 4. [ka] 7.3.2.11 Scaling List Data Syntax

[0156] [Table 19]

[0157] ScalingFactor[6][ma The trixId][][] is derived as follows: [ka] Also, [ka] ScalingFactor[6][6][matrixId][0][0]=sc aling_list_dc_coef_minus8[2][matrixId]+8 (7-XX) with matrixId=0,3 / / If zeroing Quantization matrices with rectangular sizes are zeroed for samples that satisfy the following conditions: [ka] - The decoded tu is not encoded by the default transformation mode (1< <size IdW)==32, x>16 - The decoded tu is not encoded in the default transformation mode (1< <sizeI dH)==32, y>16

[0158] 5.4. Scaling Matrix Implementation #4 This chapter provides an example that does not allow scaling matrices for QR-BDPCM coded blocks. Provide. [ka] 8.7.3 Scaling of Conversion Coefficients The inputs to this process are: - The top-left sample of the current luma transformation block relative to the top-left luma sample of the current picture. Luminance position (xTbY, yTbY) that specifies the pull - variable nTbW, which specifies the width of the transformation block; - variable nTbH, which specifies the height of the transformation block; - the variable cIdx, which specifies the color components of the current block, - The variable bitDepth, which specifies the bit depth of the current color components. The output of this process is a set of scaled transform coefficients (nT bW) × (nTbH) array d. The quantization parameter qP is derived as follows. - If cIdx is equal to 0, the following applies: qP=Qp'Y (8-1019) - Otherwise, if cIdx is 1, the following applies: qP=Qp'Cb (8-1020) - Otherwise (cIdx=2), the following applies: qP=Qp'Cr (8-1021) The variable rectNonTsFlag is derived as follows. rectNonTsFlag=(((Log2(nTbW)+Log2(nTbH) )&1)==1 && (8-1022) transform_skip_flag[xTbY][yTbY]==0) The variables bdShift, rectNorm, and bdOffset are derived as follows: It is served. bdShift=bitDepth+((rectNonTsFlag?8:0)+ (8-1023) (Log2(nTbW)+Log2(nTbH)) / 2)-5+dep_quant _enabled_flag rectNorm=rectNonTsFlag ? 181:1 (8-1024 ) bdOffset=(1<<bdShift)> >1 (8-1025) list levelScale[] is levelScale[ when k=0..5 k] = {40, 45, 51, 57, 64, 72}. If x=0..nTbW-1,y=0..nTbH-1, the scaled transformation To derive the conversion coefficients d[x][y], the following applies: - The intermediate magnification m[x][y] is derived as follows: - m[x][y] is set equal to 16 if one or more of the following conditions are true: do. - scaling_list_enabled_flag equals 0. - transform_skip_flag[xTbY][yTbY] is 1 do. [ka] - If not, the following applies: - m[x][y]=ScalingFactor[sizeIdW][sizeI dH][matrixId][x][y] (8-XXX) If you set sizeIdW to Log2(nTbW), set sizeIdH to Log2( nTbH) and matrixId is specified in Table 7-14. - The magnification ls[x][y] is derived as follows: - If dep_quant_enabled_flag is 1, the following applies: do. ls[x][y]=(m[x][y]*levelScale[(qP+1)%6] )<<((qP+1) / 6) (8-1026) - Otherwise (dep_quant_enabled_flag is equal to 0) , the following applies: ls[x][y]=(m[x][y]*levelScale[qP%6])<<( qP / 6) (8-1027) The values ​​dnc[x][y] are derived as follows: dnc[x][y] = (8-1028) (TransCoeffLevel[xTbY][yTbY][cIdx][x][ y]*ls[x][y]*rectNorm+bdOffset)>>bdShift The scaled transform coefficients d[x][y] are derived as follows: d[x][y]=Clip3(CoeffMin,CoeffMax,dnc[x] [y]) (8-1029)

[0159] 5.5. Transformation Skip Flag Semantics Implementation #5 transform_skip_flag[x0][y0] is for the luminance transformation block. The array index x0,y0 specifies whether to apply the transformation to the left The position (x0, y0). transform_skip_flag[x0][y0]=1 means Specifies that no transformation should be applied to the luminance transformation block. flag[x0][y0]=0 determines whether to apply the transformation to the luminance transformation block. Specifies a dependency on other syntax elements. [ka]

[0160] FIG. 11 is a block diagram of a video processing device 1100. The device 1100 is The device 1100 may be used to implement one or more of the methods described herein. , tablets, computers, IoT (Internet of Things) receivers, etc. The device 1100 may include one or more processing units 1102, one or more memories 1104, and , and video processing hardware 1106. 02 may be configured to implement one or more of the methods described herein. The library(s) 1104 may include a library for implementing the methods and techniques described herein. It may also be used to store data and code used in the video processing hardware. The controller 1106 is used to implement the techniques described herein in hardware circuits. This is also fine.

[0161] In the embodiments of the present invention, the following solutions can be implemented as preferred solutions.

[0162] The following solutions combine the additional techniques described in the previous chapter (e.g., item 1) Both may be implemented.

[0163] 1. A coding model for converting between a coded representation of a video block of a video and said video block. Whether or not the skip transform mode is enabled for the transform based on the transform information. (1202) determining whether the and in the transform skip mode, during the transform,

[0023] In one embodiment, the method includes: skipping the application of a transform to at least some coefficients representing For example, method 1200 shown in FIG.

[0164] 2. Block Differential Pulse Code Modulation (BDPCM) or Quantized Residual BDPCM (QR- Transform skip mode is enabled due to the coding mode information indicating BDPCM (Binary Digital Positive Transformation Coding) mode. The method according to solution 1, wherein the method is determined to be

[0165] 3. A flag in the encoded representation indicating the transform skip mode is not parsed. Yes, the method described in Solution 1.

[0166] 4. The flag indicating transformation skip mode in the encoded representation is omitted from parsing. , the method described in Solution 1.

[0167] The following solutions combine the additional techniques described in the previous chapter (e.g., item 2). Both may be implemented.

[0168] 5. Block Differential Pulse Code Modulation (BDPCM) or Quantized Residual BDPCM (QR-B The coded representation of the video block and the video block are converted using the DPCM (Digital Phase Change Multiplication) mode. determining a scaling matrix to be used for conversion to and from the block; and performing the transformation using a scaling matrix, the scaling matrix being is used to scale at least some of the coefficients that represent this video block to A video processing method.

[0169] 6. The transform has one mode depending on the number of transforms applied to the coefficients during the transform. 6. The method of claim 5, comprising applying the scaling matrix based on

[0170] 7. Block Differential Pulse Code Modulation (BDPCM) or Quantized Residual BDPC for Transform Due to the use of M(QR-BDPCM) mode, the coding representation of the video block and the determining to disable use of a scaling matrix for conversion to and from the scale; and performing said transformation using a scaling matrix, the scaling matrix being a function of the used to scale at least some of the coefficients representing the image block. Processing methods.

[0171] The following solutions combine the additional techniques described in the previous chapter (e.g., item 3). Both may be implemented.

[0172] 8. A conversion scheme for converting between a coded representation of a video block of a video and said video block. The in-loop filter is selected depending on whether loop mode is enabled for the transform. determining applicability of the in-loop filter; and based on the applicability of the in-loop filter, and performing a conversion of the image in the conversion skip mode. skipping applying a transform to at least some coefficients representing an image block; Processing methods.

[0173] 9. The method according to solution 8, wherein the in-loop filter comprises a deblocking filter. Law.

[0174] 10. A transform skip mode of a video block and another transform skip mode of an adjacent block The method of any one of claims 8 to 9, further comprising determining a strength of an in-loop filter based on the code. Any method as described above.

[0175] 11. The determining step comprises determining whether the transform skip mode is disabled for the video block. determining that the in-loop filter is not applicable because the in-loop filter is 10. The method according to any one of Solutions 8-9.

[0176] The following solutions incorporate the additional steps described in the previous chapter (e.g., steps 4 and 5). The present invention may be implemented in conjunction with the techniques described above.

[0177] 12. An interface is provided for converting between a video block of a video and a coded representation of this video block. The same scaling row is used for transformations based on intra-block copying and intra-block copy coding. selecting a scaling matrix such that a column is selected; performing the transformation using a scaling matrix, the scaling matrix being 23. A video processing method according to claim 22, wherein the video processing method is used to scale at least some of the coefficients of the block.

[0178] The following solutions combine the additional techniques described in the previous chapter (e.g., item 6). Both may be implemented.

[0179] 13. For converting between video blocks of a video and a coded representation of said video, selecting a scaling matrix based on the selected transformation matrix; and performing the transformation using a scaling matrix, the scaling matrix being used to scale at least some coefficients of the image block, A transformation matrix is ​​used to transform at least some of the coefficients of the video block during the transformation. The image processing device used.

[0180] 14. Selecting the scaling matrix determines whether the transformation of the image block is a subblock. The method according to Solution 13 is based on whether you use the .md5 encoding mode or not.

[0181] 15. The method according to solution 14, wherein the sub-block coding mode is an affine coding mode. The method described.

[0182] 16. The scaling matrix for the affine coding mode is Do not use different scaling matrices for different image blocks, as described in Solution 15. How to.

[0183] The following solutions combine the additional techniques described in the previous chapter (e.g., item 7). Both may be implemented.

[0184] 17. For converting between a video block of a video and a coded representation of said video block, Selecting a scaling matrix based on a secondary transformation matrix selected for the transformation. and performing the transformation using the selected scaling matrix, The scaling matrix is ​​for scaling at least some of the coefficients of the image block. the secondary transformation matrix being used to transform at least some of the image blocks during the transformation A video processing method used to transform some residual coefficients.

[0185] 18. The secondary transformation matrix is ​​applied to an M×N upper left portion of the image block, The scaling matrix is ​​applied to a portion of the image block that is greater than the M×N upper left portion. The method described in Solution 17.

[0186] 19. The secondary transformation matrix is ​​applied to an M×N upper left portion of the image block, The scaling matrix is ​​applied only to the M×N top-left portion of the image block. 7. The method according to claim 7.

[0187] 20. Solution 17, wherein a syntax element in the encoded representation indicates the scaling matrix. A method according to any one of claims 1 to 19.

[0188] The following solutions combine the additional techniques described in the previous chapter (e.g., item 8). Both may be implemented.

[0189] 21. For video blocks with non-square shapes, determining a scaling matrix for use in converting to and from the encoded representation, A syntax element in signals the scaling matrix. and performing the transformation based on a ring matrix, the scaling matrix being used to scale at least some coefficients of the image block during conversion. A video processing method.

[0190] 22. The syntax element is a scaling matrix of a previous square block. 22. The method of solution 21, for predictively coding the matrix.

[0191] The following solutions combine the additional techniques described in the previous chapter (e.g., item 9). Both may be implemented.

[0192] 23. When converting between the coded representation of a video block and the video block, a scaling matrix is ​​used as a part. determining whether the first set of positions is to be applied to the image block; The scaling matrix is ​​applied and the remaining positions are disabled. and performing a transformation by partially applying the scaling matrix to the image. Image processing methods.

[0193] 24. The first set of locations includes M*N locations at the top left of the image block. The method according to solution 23.

[0194] 25. The first set of locations includes M*N locations at the top of the image block. The method according to solution 23.

[0195] 26. The first set of locations includes M*N locations to the left of the image block. The method described in Measure 23.

[0196] The following solutions are based on the solutions described in the previous chapter (e.g., items 10 and 11). It may be implemented with additional techniques.

[0197] 27. When converting between the coded representation of a video block and the video block, a scaling matrix is determining that it applies; and performing the transformation based on the scaling matrix, The encoded representation signals the number of elements of the scaling matrix, the number being indicative of the transformation. 6,233,311; and 6,233,311. A video processing method that relies on the application of zeroing coefficients in a video signal.

[0198] 28. The transformation sets all positions except the top left M×N positions of the image block to zero. 28. The method according to claim 27, further comprising:

[0199] The following solutions incorporate additional techniques described in the previous chapter (e.g., item 11). It may also be implemented in conjunction with

[0200] 29. The method according to any one of solutions 27 to 28, wherein the number depends on the transformation matrix used during the transformation. method.

[0201] 30. The transformation matrix is ​​of size K×L, and only the top M×N coefficients are zeroed. The method described in Solution 29.

[0202] 31. The scaling matrix is ​​subsampled by a factor determined from K or L. The method according to any of Solutions 27 to 30, wherein the method is applied by

[0203] The following solutions incorporate additional techniques described in the previous chapter (e.g., item 12). It may also be implemented in conjunction with

[0204] 32. When converting between a video block and a coded representation of said video block, a particular type of video determining a single quantization matrix to use based on the size of the image block; performing a transformation using a matrix.

[0205] 33. The size of the image block is smaller than W×H, where W and H are integers. The method described in Solution 32.

[0206] 34. The method according to any one of solutions 32 to 33, wherein the width of the image block is smaller than a threshold value. How to.

[0207] 35. The method according to any one of solutions 32 to 33, wherein the height of the image block is smaller than a threshold value. Method of posting.

[0208] 36. The quantization matrix is ​​an identity quantization matrix that does not affect the quantization value. Solution 32. The method according to claim 32.

[0209] 37. Solutions 1 to 3, wherein the transformation includes encoding the video to the encoded representation. 7. A method according to any one of claims 6 to 6.

[0210] 38. The transform includes decoding the encoded representation to generate pixel values ​​of the image. 37. The method according to any of Solutions 1 to 36, comprising:

[0211] 39. A processing device configured to implement the method according to one or more of solutions 1 to 38. A video decoding device comprising:

[0212] 40. A processing device configured to implement the method according to any one or more of solutions 1 to 38. The video encoding device includes:

[0213] 41. A computer program product having computer code stored therein, The code, when executed by a processing device, causes the processing device to perform a process according to any one of solutions 1 to 38. A computer program product implementing the method described herein.

[0214] 42. Any method, apparatus or system described herein.

[0215] Some embodiments of the disclosed technology may include a method for enabling video processing tools or modes. In one example, determining or judging that a video processing tool or mode is enabled. When a video block is processed, the encoder uses this tool or mode to use or implement a tool, but results in The resulting bitstream does not necessarily need to be modified; i.e., the block of video The conversion from the input video to a bitstream representation is made by a video processing tool or Use this video processing tool or mode if the tool or mode is enabled. When a video processing tool or mode is enabled in The bitstream is modified based on the video processing tool or mode. i.e., the video processing tools or The mode is used to convert from a bitstream representation of the image to blocks of the image.

[0216] Some embodiments of the disclosed technology include disabling video processing tools or modes. In one example, determining or judging that a video processing tool or mode is disabled. When enabled, the encoder converts blocks of video into a bitstream representation of the video. In another example, you may not use this tool or mode to If the mode is disabled, the decoder will You know that the bitstream has not been modified using any video processing tools or modes. The bitstream is then processed.

[0217] FIG. 15 is a block diagram illustrating an example video encoding system 100 that can utilize techniques of this disclosure. As shown in FIG. 15, the video encoding system 100 includes a source device 110. and a destination device 120. The source device 110 may include a video encoder. Destination device 120 generates encoded video data, which may also be referred to as a source device. A video decoding device may be used to decode the encoded video data generated by the video decoder 110. The source device 110 may include a video source 112, a video encoder 114, and input / output and an input / output (I / O) interface 116.

[0218] The video source 112 may be a source such as a video capture device, a video content provider, or the like. An interface for receiving video data from a video recorder and / or a video recorder for generating video data. a computer graphics system to create a The video data may include one or more pictures. 14 encodes video data from a video source 112 to generate a bitstream. A bitstream may include a sequence of bits that form a coded representation of video data. A bitstream may contain coded pictures and associated data. A coded picture is a coded representation of a picture. The associated data is a sequence of It may include parameter sets, picture parameter sets, and other syntax structures. The I / O interface 116 may include a modulator / demodulator (modem) and / or a transmitter. The encoded video data is transmitted to the I / O interface 130 via the network 130a. The encoded video can then be sent directly to the destination device 120 via the video source 116. The image data is stored on the storage medium / server 130b for access by the destination device 120. It may be memorized.

[0219] The destination device 120 includes an I / O interface 126, a video decoder 124, and and a display device 122.

[0220] The I / O interface 126 may include a receiver and / or a modem. The O interface 126 is connected to the source device 110 or the storage medium / server 130b. The video decoder 124 may obtain the encoded video data from the The display device 122 may then display the decoded video data to the user. The display device 122 may be integrated with the destination device 120 or may be external. external to the destination device 120 configured to interface with a display device; It's fine.

[0221] The video encoder 114 and the video decoder 124 are based on the High Efficiency Video Coding (HEVC) standard. standards, the Universal Video Coding (VVVM) standard, and other current and / or future standards. It may operate in accordance with the video compression standard.

[0222] FIG. 16 is a block diagram illustrating an example of a video encoder 200. 200 may be the video encoder 114 in the system 100 shown in FIG. stomach.

[0223] The video encoder 200 may be configured to perform any or all of the techniques of this disclosure. In the embodiment of FIG. 16, the video encoder 200 may include a number of functional modules. The techniques described in this disclosure provide for a method for implementing a video encoder 200 that In some examples, the processing unit may be shared by any of the techniques described in this disclosure. may be configured to do all of the above.

[0224] The functional modules of the video encoder 200 include a division unit 201 and a mode selection unit 202. 203, a motion estimation unit 204, a motion compensation unit 205 and an intra prediction unit A prediction unit 202 may include a residual generation unit 207 and a transform unit 208. 208, a quantization unit 209, an inverse quantization unit 210, and an inverse transform unit 21 1, a reconstruction unit 212, a buffer 213, and an entropy coding unit 214. and

[0225] In other examples, video encoder 200 may have more, less, or different functionality. In one example, the prediction unit 202 may include an intra-block An IBC unit may contain at least one reference copy (IBC) unit. In IBC mode, the picture in which the current video block is located is the prediction (p redication) can be carried out.

[0226] In addition, some components such as a motion estimation unit 204 and a motion compensation unit 205 are The components may be highly integrated, but for purposes of illustration, they are shown separately in the example of FIG. It is represented.

[0227] The division unit 201 can divide a picture into one or more video blocks. The video encoder 200 and the video decoder 300 can handle various video block sizes. May be supported.

[0228] The mode selection unit 203 may, for example, select intra or inter based on the error result. and select one of the coding modes for the resulting intra- or inter-coded block. The residual block data is fed to the residual generation unit 207 to generate the residual block data, and the reconstruction unit Alternatively, the reference picture may be provided to the block 212 to reconstruct the coding block as a reference picture. In an embodiment, the mode selection unit 203 is configured to select an inter prediction signal and an intra prediction signal. A combination of intra- and inter-prediction (CIIP) modes for predicting based on the signal In addition, the mode selection unit 203 may select the block of the inter prediction mode. Select the resolution of the motion vectors (e.g., sub-pixel or integer pixel accuracy) to Good too.

[0229] A motion estimation unit 204 for performing inter prediction on the current video block. The method includes comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may generate motion information for the current video block. is a picture from buffer 213 other than the picture associated with the current video block. A prediction video block is generated for the current video block based on the motion information and the decoded samples. The check may be determined.

[0230] The motion estimation unit 204 and the motion compensation unit 205 may, for example, The current slice is selected based on whether the block is an I slice, a P slice, or a B slice. Different operations may be performed on the current image block.

[0231] In some examples, the motion estimation unit 204 may perform a unidirectional motion estimation for the current video block. For the current video block, motion estimation unit 204 performs forward prediction using list 0 or may search the reference pictures in list 1 to find the reference video block. The estimation unit 204 estimates the reference video block, the current video block, and the reference video block. and a motion vector indicating the spatial displacement between the reference pixels in list 0 or list 1. The motion estimation unit 204 may generate a reference index that indicates the frame. The motion information of the current video block is calculated by dividing the motion vector by the prediction index, prediction direction indicator, and motion vector. The motion compensation unit 205 may output the motion information of the current video block as A prediction video block for the current block may be generated based on the reference video block indicated by .

[0232] In another example, the motion estimation unit 204 may bi-directionally predict the current video block. Preferably, the motion estimation unit 204 selects a reference picture from the list 0 for the current video block. A reference picture block may be searched for to lock, and the reference picture in List 1 may be searched for to lock. , and The motion estimation unit 204 estimates the reference video blocks in list 0 and list 1. A reference index indicating the reference picture and a distance between the reference video block and the current video block. The motion estimation unit 204 may generate a motion vector that indicates the spatial displacement of the current The reference index and motion vector of the video block are calculated based on the motion information of the current video block. The motion compensation unit 205 may output the motion information of the current video block as A prediction block for the current video block may be generated based on the reference video block. stomach.

[0233] In some examples, the motion estimation unit 204 may include, for the decoding process of the decoder: The full set of motion information may be output.

[0234] In some examples, the motion estimation unit 204 may generate a full set of motion information for the current picture. Instead, motion estimation unit 204 may output a motion vector of another video block. The motion information of the current video block may be signaled by referring to the motion information. The estimation unit 204 estimates the motion information of the current video block based on the motion information of the neighboring video blocks. may be determined to be sufficiently similar to

[0235] In one example, the motion estimation unit 204 may generate a syntax In the structure, the current video block has the same motion information as another video block. The value shown to the image decoder 300 may be indicated.

[0236] In another example, the motion estimation unit 204 may In the sentence structure, another video block and a motion vector difference (MVD) may be identified. The motion vector difference is the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 calculates the difference between the motion vector of the indicated video block and the motion vector of the indicated video block. The motion vector of the current image block is determined by using the vector and the difference of the motion vector. Good too.

[0237] As mentioned above, the video encoder 200 may predictively signal motion vectors. Two examples of predictive signaling techniques that may be implemented by video encoder 200 are: Includes adaptive vector prediction (AMVP) and merge mode signaling.

[0238] The intra prediction unit 206 performs intra prediction on the current video block. If intra prediction unit 206 intra predicts the current video block, The traprediction unit 206 uses the decoded samples of other video blocks in the same picture. Based on the prediction, the prediction data for the current video block may be generated. The prediction data for the block includes a predicted video block and various syntax elements. Good too.

[0239] The residual generation unit 207 generates a predicted residual of the current video block from the current video block. By subtracting the image block (e.g., indicated by a minus sign), Residual data may be generated for the current video block. The residual image data corresponds to different sample components of the samples in the current image block. It may include blocks.

[0240] In another example, e.g., in skip mode, the residual for the current video block The data may be missing and the residual generation unit 207 may not perform the subtraction operation.

[0241] The transform processing unit 208 performs the transform on the residual video block associated with the current video block. Applying one or more transformations to the current video block A number of image blocks may be generated.

[0242] The transform processing unit 208 generates a transform coefficient image block associated with the current image block. After generating, the quantization unit 209 quantizes one or more The transformation coefficients associated with the current video block are calculated based on the quantization parameter (QP) value of the Several image blocks may be quantized.

[0243] The inverse quantization unit 210 and the inverse transform unit 211 apply inverse quantization to the transform coefficient image block. and applying the inverse transform to reconstruct a residual image block from the transform coefficient image block. The reconstruction unit 212 may generate one or more predictions generated by the prediction unit 202. Add the reconstructed residual image block to the corresponding samples from the image block and subtract the current image block from the corresponding samples. The reconstructed image block associated with the block may be generated and stored in the buffer 213. Cut.

[0244] After the reconstruction unit 212 reconstructs the image block, To reduce filtering artifacts, a loop filtering operation may be performed.

[0245] The entropy encoding unit 214 is a component of the video encoder 200. The entropy encoding unit 214 may receive data from performing one or more entropy encoding operations to generate entropy encoded data; A bitstream containing the entropy encoded data may be output.

[0246] FIG. 13 is a block diagram showing an example of a video decoder 300. 0 may be the video decoder 114 in the system 100 shown in FIG.

[0247] The video decoder 300 is configured to perform any or all of the techniques of this disclosure. In the embodiment of FIG. 13, the video decoder 300 includes a number of functional modules. The techniques described in this disclosure are shared among various modules of the video decoder 300. In some examples, the processing device may include any or all of the techniques described in this disclosure. The above configuration may be adopted.

[0248] In the embodiment of FIG. 13, the video decoder 300 includes an entropy decoding unit 301. , a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, The image processing system includes a conversion unit 305, a reconstruction unit 306, and a buffer 307. The image decoder 300 may, in some examples, be described with respect to the video encoder 200 (FIG. 16). A decoding pass may be performed that is roughly the reverse of the encoding pass previously performed.

[0249] The entropy decoding unit 301 retrieves the coded bit stream. The bit stream is the entropy-encoded video data (e.g., The entropy decoding unit 301 may include an entropy encoding block. The entropy-decoded image data is decoded, and a motion compensation unit is generated from the entropy-decoded image data. The frame 302 stores the motion vector, the motion vector precision, the reference picture list index, and The motion compensation unit 302 may determine motion information including, for example, Such information may be determined by implementing AMVP and merge modes.

[0250] The motion compensation unit 302 may generate a motion compensated block, possibly performs interpolation based on an interpolation filter. The syntax element includes The interpolation filter may include an identifier for the interpolation filter to be used.

[0251] Motion compensation unit 302 is used by video encoder 20 during the encoding of a video block. Interpolation for sub-integer pixels of the reference block using an interpolation filter such as The motion compensation unit 302 may calculate a motion vector based on the received syntax information. The encoder 200 determines the interpolation filter to be used, and uses the interpolation filter to generate the predicted block. A check may be generated.

[0252] The motion compensation unit 302 uses a portion of the syntactic information to estimate the motion of the encoded video sequence. The blocks used to encode the frame(s) and / or slice(s). The size of the block, how each macroblock of a picture in a coded video sequence is partition information describing how the image is divided into partitions; modes indicating how each partition is coded; one or more reference frames (and reference frame lists) between each of the inter-coded blocks; and other information for decoding the encoded video sequence.

[0253] The intra prediction unit 303 may, for example, predict intra-frames received in a bitstream. Prediction modes may be used to form a prediction block from spatially adjacent blocks. The inverse quantization unit 303 is provided to the bitstream, and the entropy decoding unit The quantized video block coefficients decoded by 301 are dequantized (i.e., The inverse transform unit 303 applies the inverse transform.

[0254] The reconstruction unit 306 combines the residual block with the motion compensation unit 202 or the intra prediction unit. The decoded block is summed with the corresponding prediction block generated by unit 303 to obtain a decoded block. If desired, a block may be formed after decoding to remove block artifacts. A deblocking filter may be applied to filter the deblocked blocks. The decoded video blocks are stored in a buffer 307, which stores the video blocks for subsequent Provides reference blocks for compensation / intra prediction and decodes the data for display on a display device. Generates an encoded image.

[0255] As used herein, the term "video processing" includes video encoding, video decoding, video compression, and For example, a video compression algorithm converts a pixel representation of an image into It may be applied during conversion to a corresponding bitstream representation or vice versa. A bitstream representation of a current video block may be, for example, as specified by the syntax: This may correspond to bits spread to the same or different locations in the bitstream. For example, a video block may be a set of 10 ... using bits in the header and other fields in the bitstream Furthermore, during conversion, the decoder may With the knowledge that some fields may or may not be present based on the Similarly, an encoder may parse the bitstream using a particular syntax field. Determines whether the syntax field should be included or not The coding is done accordingly by including or excluding A coded representation may be generated.

[0256] FIG. 14 is an exemplary video processing system in which various techniques disclosed herein may be implemented. 2000. Various implementations of the modules of system 2000. The system 2000 may include an input for receiving video content. The video content may include a video output unit 2002. The video content may be in a raw or uncompressed format. For example, the signal may be received as 8 or 10 bit multi-component pixel values, or compressed. The input unit 2002 may receive the signal in a compressed or encoded format. may represent a memory interface, a peripheral bus interface, or a storage interface. Examples of network interfaces are Ethernet, passive optical network, Wired interfaces such as Passive Optical Network (PON) and Wi-Fi or This includes wireless interfaces such as ruler interfaces.

[0257] The system 2000 may implement various encoding or encoding methods described herein. The encoding component 200 may include an encoding component 2004 that is capable of 4 is a component that encodes the average bit rate of the video from the input unit 2002. 04 to produce a coded representation of the video. This is sometimes called video compression or video transcoding technology. The output of 4 may be stored or connected, as represented by component 2006. The input unit 2002 may transmit the recorded information via a connected communication. The bitstream (or coded) representation of a stored or communicated video may consist of components 2008 and transmits pixel values ​​or table values ​​to the display interface 2010. A user-viewable image may be generated from the bitstream representation. The process of generating an image is sometimes called image expansion. The processing operations are referred to as "encoding" operations or tools, and encoding tools or operations include encoders and and corresponding decoding tools or operations that reverse the result of the decoding performed by the decoder. It will be understood that this is done in a manner that

[0258] An example of a peripheral bus interface or display interface is the Universal Serial Bus (USS). Bus (USB) or High-Definition Multimedia Interface (HDMI®) or a display port, etc. Examples of storage interfaces include a serial Advanced Technology Attachment (SATA), PCI, IDE interface The technology described in this specification is applicable to mobile phones, laptops, smartphones, etc. or other devices capable of digital data processing and / or image display. The present invention may be implemented in any suitable electronic device.

[0259] FIG. 17 is a flow chart illustrating an example method 1700 of video processing. 00 includes converting (1702) between video blocks of video and a coded representation of the video. , the encoded representation conforms to a format rule, the format rule being The applicability of the transform skip mode to a video block depends on the coding conditions of the video block. The format rule specifies whether or not the conversion skip mode is used. a syntax element representing a possibility is omitted from the encoded representation, and the transform skip mode is applying a forward transform to at least some of the coefficients prior to encoding into said coded representation. and, upon decoding, performing a small inverse transform before decoding from the encoded representation. and skipping application of the method to at least some of the coefficients.

[0260] FIG. 18 is a flow chart illustrating an example method 1800 of video processing. 00 for transforming between two adjacent video blocks of an image and a coded representation of the image, Depending on whether a forward or inverse transform is used for the transformation, the in-loop filter and the re-transformation determining (1802) which of the constructed filters is to be used for the conversion; A forward transform may be implemented by applying the forward transform to at least some of the input data before encoding the encoded representation. and upon decoding, skipping the application of the quantifier to the coded representation before decoding the coded representation. skipping applying the inverse transform to at least some of the coefficients in the previous step; The conversion is based on the use of an in-loop filter or a post-reconstruction filter (18). 04) and more.

[0261] FIG. 19 is a flow chart illustrating an example method 1900 of video processing. 00 for converting between a video block of a video and a coded representation of the video The Block Differential Pulse Code Modulation (BDPCM) encoding tool used in the conversion The scaling tool is based on the quantized residual BDPCM (QR-BDPCM) coding tool. (1902) and use the scaling tool to determine the transformation. and performing (1904) a syntax element in the encoded representation, the syntax element being The use of the scaling tool represents the representation of the video block during encoding. scaling at least some of the coefficients or removing some of the coefficients from the coded representation upon decoding. This involves descaling at least some of the coefficients.

[0262] FIG. 20 is a flow chart illustrating an example method 2000 of video processing. 00 is a video block conversion for converting between video blocks of video and a coded representation of the video. Block Differential Pulse Code Modulation (BDPCM) coding tool or quantized residual BD Disabled the use of scaling tools due to PCM (QR-BDPCM) encoding tools (2002) and converting without using a scaling tool. (2004), in which the use of the scaling tool represents a video block during encoding. scaling at least some of the coefficients or removing some of the coefficients from the coded representation upon decoding. This involves descaling at least some of the coefficients.

[0263] FIG. 21 is a flow chart illustrating an example method 2100 of video processing. 00 is a block diagram of a video block of an image that is used for conversion between a video block of an image and a coded representation of an image. selecting (2102) a scaling matrix based on the selected transformation matrix, The scaling matrix scales at least some of the coefficients of the image block. the transformation matrix being used to transform at least some of the image blocks during the transformation; The scaling matrix is ​​used to transform the coefficients. and performing the conversion (2104).

[0264] FIG. 22 is a flow chart illustrating an example method 2200 of video processing. 00 means that the secondary transformation matrix has been applied to some of the image blocks of the image according to the rules. determining whether to apply a scaling matrix based on whether the ) wherein the scaling matrix scales at least some of the coefficients of the image block. The secondary transformation matrix is ​​used to reduce the portion of the image block during transformation. Also, some residual coefficients are transformed using the selected scale. A conversion matrix is ​​used to convert between the image block of the image and the bitstream representation of the image. (2204)

[0265] FIG. 23 is a flow chart illustrating an example method 2300 of video processing. 00 for image blocks having non-square shapes, Determining a scaling matrix to use for converting to and from a coded representation of the video (2302 ), wherein a syntax element in the coded representation signals the scaling matrix; The scaling matrix scales at least some of the coefficients of the video block during the transformation. Determining, which is used to scale; and performing the transformation based on the scaling matrix (2304).

[0266] FIG. 24 is a flow chart illustrating an example method 2400 of video processing. 00 includes converting (2402) between video blocks of a video and a coded representation of the video. A video processing method comprising: The image block is transformed based on a rule to which the scaling matrix is ​​applied. A first number of positions and a second number of positions to which no scaling matrix is ​​applied during the transformation. nothing.

[0267] FIG. 25 is a flow chart illustrating an example method 2500 of video processing. 00 indicates that a scaling matrix is ​​applied when converting between a video block of an image and a coded representation of the image. determining whether the information is to be used (2502); performing the transformation based on the scaling matrix, The coded representation indicates the number of elements of the scaling matrix, the number indicating the coefficients to be zeroed. It depends on whether it is applied to the coefficients of the image block.

[0268] FIG. 26 is a flow chart illustrating an example method 2600 of video processing. 00 performs conversion between video blocks of a video and a coded representation of said video according to rules. A video processing method including (2602), The image block is transformed by applying a K×L transformation matrix to transform the coefficients of the image block. In the coded representation after zeroing all the transform coefficients except for the top left M×N transform coefficient, Expressed, The encoded representation is a representation of the scaling matrix elements at positions corresponding to the zeroing. configured to eliminate signaling; The scaling matrix is ​​used to scale the transform coefficients.

[0269] FIG. 27 is a flow chart illustrating an example method 2700 of video processing. 00 is the number of bits of a video block when converting between a video block of a video and a coded representation of the video. A rule-based determination of whether to use a single quantization matrix based on size. (2702), wherein all image blocks having the size are quantized in the single quantization row. Using the sequence, determining performing the transform using the quantization matrix.

[0270] The following three sections describe example image processing techniques, which are numbered as follows:

[0271] Section A:

[0272] 1. Performing a conversion between a video block of a video and a coded representation of the video, The representation conforms to a format rule, the format rule being in accordance with the video block. The applicability of the conversion skip mode to the block is determined by the coding condition of the video block. The format rule specifies whether the conversion skip mode is available. a syntax element that is omitted from the encoded representation, and the transform skip mode is It is also possible to skip applying a forward transform to at least some of the coefficients before encoding them into a linear representation. and, upon decoding, performing at least an inverse transform before decoding from said encoded representation. and skipping the application to some coefficients.

[0273] 2. Block Differential Pulse Code Modulation (BDPCM) or Quantized Residual BDPCM (QR- A video block coding scheme indicating that BDPCM (Binary Digital Persistent Coding) is used for the video block. 2. The method of example 1, wherein the determination is made to enable the transform skip mode due to a condition.

[0274] 3. Non-block differential pulse code modulation (Non-BDPCM) or non-quantized residual BDPCM ( A video block indicating that non-QR-BDPCM (non-QR-BDPCM) is used for the video block. The first embodiment is a case where it is determined that the transform skip mode is disabled due to the encoding condition. Law.

[0275] 4. For converting between two adjacent image blocks of an image and a coded representation of said image, Depending on whether a directional or inverse transform is used for the transformation, the in-loop filter and the post-reconstruction the forward transform includes determining which of the filters to use for the transform, applying said forward transform to at least some of the coefficients before encoding them into a coded representation. and upon decoding, performing the inverse transform before decoding from the encoded representation. skipping application of the in-loop filter to at least some of the coefficients; and performing said transformation based on the use of a post-reconstruction filter or a post-reconstruction filter. .

[0276] 5. The forward transform or the inverse transform is in a transform skip mode or a block differential parity mode. Quantized residual BDPCM (QR-BDPCM), or The in-loop filter or The use of a post-reconstruction filter eliminates the need for transform skip for the two adjacent image blocks. Mode, or Block Differential Pulse Code Modulation (BDPCM), or Quantized Residual BDPC M(QR-BDPCM), or palette mode as described in Example 4. Method of posting.

[0277] 6. The forward transform or the inverse transform includes a transform skip mode, and a boundary filter The derivation of the filtering strength includes applying the transform scaling to one or both of two adjacent image blocks. The method described in Example 4 depends on one or more syntax elements that indicate whether the hopping mode is enabled. Method of posting.

[0278] 7. The forward transform or the inverse transform includes a transform skip mode, and two adjacent In response to a sample located in a video block being encoded in the transform skip mode, The deblocking filter, the sample adaptive offset, the adaptive loop filter, or the above The method of example 4, wherein the post-reconstruction filter is disabled.

[0279] 8. The forward transform or the inverse transform includes a transform skip mode, and two adjacent In response to the transform skip mode being enabled for a video block, The loop filter and the post-reconstruction filter are adapted to detect an edge between two adjacent image blocks. The method described in Example 7 does not apply to.

[0280] 9. The forward transform or the inverse transform includes a transform skip mode, and two adjacent in response to the transform skip mode being enabled for one of the video blocks. The in-loop filter and the post-reconstruction filter are configured to filter two adjacent image blocks. The method described in Example 7 does not apply to inter-group samples.

[0281] 10. The forward transform or the inverse transform includes a transform skip mode, and two adjacent In response to the transform skip mode being enabled for a corresponding video block, Filter the samples using a filter other than a loop filter or a post-reconstruction filter. The method of Example 4, wherein the ligation is performed.

[0282] 11. The method of example 10, wherein the filter comprises a smoother filter.

[0283] 12. The image is modulated by Pulse Code Modulation (PCM) or Block Differential Pulse Code Modulation (B Video blocks coded using DPCM (DPCM) or quantized residual BDPCM (QR-BDPCM) or another type of mode that does not apply a forward or inverse transform to the video block. and wherein the in-loop filter or the post-reconstruction filter is Whether or not a video block is used for conversion varies with respect to the two adjacent video blocks. When the conversion skip mode is enabled, the same The method of Example 4, wherein the antibody is determined as follows.

[0284] 13. The method of any one of embodiments 4 to 12, wherein the in-loop filter comprises a deblocking filter. Any method as described above.

[0285] 14. An embodiment in which the post-reconstruction filter comprises a bilateral filter or a diffusion filter. The method according to any one of 4 to 12.

[0286] 15. The method of claim 1, wherein the transforming comprises encoding the video block into the encoded representation. The method according to any one of Examples 1 to 14.

[0287] 16. The transform includes decoding the encoded representation to generate pixel values ​​for the video block. The method according to any one of Examples 1 to 14, comprising:

[0288] 17. A processing device configured to implement the method according to any one or more of the first to sixth embodiments. A video decoding device comprising:

[0289] 18. A computer program product having computer code stored therein, When the code is executed by a processing device, the processing device performs the processing described in any one of Examples 1 to 17. Implement the method.

[0290] Section B:

[0291] 1. For converting between a video block of an image and a coded representation of the image, determining a scaling factor based on the encoding mode; and and performing a conversion using said scaling tool during encoding. scaling at least some of the coefficients representing the video block of and descaling at least some of the coefficients from the coded representation. Processing methods.

[0292] 2. Block Differential Pulse Code Modulation (BDP) is used for the conversion of image blocks. CM) encoding tool or Quantized Residual BDPCM (QR-BDPCM) encoding tool and determining a scaling tool factor based on a predefined value in response to the The method of Example 1, comprising:

[0293] 3. Video block to which the BDPCM encoding tool or QR-BDPCM encoding tool is applied The scaling tool multiple used to lock the image to which the transformation skip mode is applied. The transform skip mode is the same as that used for the block encoding to the encoded representation. skipping the application of a forward transform to at least some coefficients before During decoding, an inverse transform is applied to at least some of the coefficients before decoding from the coded representation. The method of example 2, comprising skipping

[0294] 4. The transform is applied to at least some coefficients of the video block during the transform. determining a multiple of the scaling tool based on one or more transformations performed on the scale tool; The method described in Example 2.

[0295] 5. The one or more transforms applied to the at least some coefficients of the image block. In response to the conversion, the scaling tool is enabled for the conversion. method.

[0296] 6. In response to one or more transforms applied to at least some coefficients of the video block. The technique for determining the scaling factor is used for intra-coded blocks. The method described in Example 4 is the same as the technique described in Example 4.

[0297] 7. Video coded using intra block copy mode and inter mode For each image block, the scaling matrix multiple is determined in the same manner as described in Example 1. Method of posting.

[0298] 8. For converting between a video block of a video and a coded representation of the video, Block differential pulse code modulation (BDPCM) coding tool for memory conversion or quantization residual Use of the scaling tool due to the differential BDPCM (QR-BDPCM) coding tool This includes determining whether to disable the scaling tool and performing the conversion without using the scaling tool. The use of the scaling tool involves at least some number of steps representing a video block when encoded. Scaling some of the coefficients or removing at least some of the coefficients from the encoded representation upon decoding. A method of video processing comprising descaling coefficients.

[0299] 9. For conversion between a video block of a video and a coded representation of a video, a selection selecting a scaling matrix based on the transformation matrix obtained by the scaling The matrix is ​​used to scale at least some of the coefficients of the image block. wherein the transformation matrix transforms at least some coefficients of the image block during the transformation. and performing a transformation using a scaling matrix. , a video processing method comprising:

[0300] 10. Selecting the scaling matrix determines whether the transformation of the image block is a default The method of example 9, based on whether or not a conversion mode is used.

[0301] 11. An embodiment in which the default transform mode includes a discrete cosine transform 2 (DCT2). 10. The method according to claim 10.

[0302] 12. The scaling matrix is ​​signaled separately for multiple transformation matrices. The method described in Example 9.

[0303] 13. Selecting the scaling matrix is ​​based on motion information of the video block. , the method described in Example 9.

[0304] 14. Selecting the scaling matrix determines whether the transformation of the image block is a subblock. The method of claim 13, wherein the method is based on whether a multi-bit encoding mode is used.

[0305] 15. The method according to embodiment 14, wherein the sub-block coding mode includes an affine coding mode. The method described.

[0306] 16. The scaling matrix for affine coding modes is different from that for non-affine coding modes. The signal transmission is performed so that the scaling matrix for the different image blocks is different from that for the different transformations used. The method of Example 15 as known in the art.

[0307] 17. Selecting the scaling matrix determines whether the image block is an affine intra-scale image. 14. The method of claim 13, wherein the encoding is based on whether the image is encoded in a predictive mode.

[0308] 18. The transform comprises encoding the video block or the video block to the encoded representation. The method according to any one of Examples 1 to 17, comprising:

[0309] 19. The transform comprises decoding the encoded representation to generate the video block or the video block. 18. The method according to any one of claims 1 to 17, further comprising generating pixel values ​​of the pixel blocks.

[0310] 20. A processing device configured to implement the method according to any one or more of the first to 19 embodiments. A video decoding device comprising:

[0311] 21. A processing device configured to implement the method according to any one or more of the first to 19 embodiments. The video encoding device includes:

[0312] 22. A computer program product having computer code stored therein, When the code is executed by a processing device, the processing device performs the processing described in any one of Examples 1 to 19. Implement the method.

[0313] Section C:

[0314] 1. A secondary transformation matrix is ​​applied to some of the image blocks of the image according to the rules. determining whether to apply a scaling matrix based on whether The scaling matrix scales at least some of the coefficients of the image block. the secondary transformation matrix is ​​used to transform the portion of the image block determining a transform coefficient used to transform at least some of the residual coefficients; The image block of the image and the image of the image are scaled using the selected scaling matrix. and converting to and from a bitstream representation.

[0315] 2. The rule is that the upper left corner of the M×N pixel of the image block contains a K×L transform block size. The scaling matrix is ​​applied to the image block in response to the secondary transformation matrix applied to the image block. 2. The method of claim 1, wherein the lock is applied to a top left portion of the M×N block.

[0316] 3. Example 1, in which the scaling matrix is ​​signaled in the bitstream representation. The method described above.

[0317] 4. The scaling matrix is ​​applied to the secondary transformation matrix or to a reduction secondary transformation matrix. Tile group headers, slide headers, pictograms for resize transformations, or rotation transformations Picture Parameter Set (PPS), Video Parameter Set (VPS), Sequence Parameter The method of embodiment 3, signaled by a meter set (SPS).

[0318] 5. The bitstream representation includes a step indicating whether to apply the scaling matrix. 1 syntax element, the bitstream representation comprising: 2. The method of claim 1, further comprising a second syntax element indicating whether

[0319] 6. The rule applies only to portions of the image block to which the secondary transformation matrix is ​​not applied. 2. The method of claim 1, further comprising applying a scaling matrix.

[0320] 7. The rule is to apply the secondary transformation matrix to the M×N upper left corner of the image block. The scaling matrix is ​​applied to the portion excluding the portion described in Example 1. How to.

[0321] 8. The rule applies the secondary transformation matrix only to a portion of the image block. The method of example 1, wherein the scaling matrix is ​​applied.

[0322] 9. For image blocks having a non-square shape, determining a scaling matrix for use in converting to and from a coded representation of the image, A syntax element in the encoded representation signals the scaling matrix and The scaling matrix scales at least some of the coefficients of the image block during the transformation. used to determine, and performing the transformation based on the scaling matrix.

[0323] 10. The syntax element is a previous scaling matrix from another scaling matrix of a previous square block of the image. 10. The method of embodiment 9, wherein the scaling matrix is ​​predictively encoded.

[0324] 11. Video processing including converting between video blocks of a video and a coded representation of said video 1. A method comprising: The image block is transformed based on a rule to which the scaling matrix is ​​applied. A first number of positions and a second number of positions to which no scaling matrix is ​​applied during the transformation. Hmm, image processing methods.

[0325] 12. The first number of locations includes M*N locations at the top left of the image block, 12. The method of embodiment 11, wherein the image block includes more than M*N locations.

[0326] 13. The first number of locations includes M*N locations at the top of the image block, 12. The method of embodiment 11, wherein the image block includes more than M*N locations.

[0327] 14. The first number of positions includes M*N positions to the left of the image block, 12. The method of embodiment 11, wherein the image block includes more than M*N locations.

[0328] 15. A scaling matrix is ​​applied when converting between an image block of an image and a coded representation of the image. determining that the method is to be used; performing the transformation based on the scaling matrix. , The coded representation indicates the number of elements of the scaling matrix, the number indicating the coefficients to be zeroed. The video processing method is dependent on whether it is applied to coefficients of the video block.

[0329] 16. For a 64x64 transform, the transform is performed on the top left MxN positions of the image block. zeroing all positions except for the number of elements of the scaling matrix; The method of example 15, wherein M / 8*N / 8.

[0330] 17. Converting between video blocks of a video and a coded representation of said video according to rules. A video processing method comprising: The image block is transformed by applying a K×L transformation matrix to transform the coefficients of the image block. In the coded representation after zeroing all the transform coefficients except for the top left M×N transform coefficient, Expressed, The encoded representation is a representation of the scaling matrix elements at positions corresponding to the zeroing. configured to eliminate signaling; The scaling matrix is ​​used to scale the transform coefficients. method.

[0331] 18. The elements of the scaling matrix located in the area outside the upper left M × N coefficients The method of example 17, wherein the signal notification is skipped.

[0332] 19. The scaling matrix is ​​scaled by a ratio determined from the K and / or the L. The method of Example 17 is applied by subsampling.

[0333] 20. The image block is divided into a number of sub-regions, each of which has a size of Uw*U h, each of which is located within a sub-region of the region containing M×N coefficients at the top left of the image block. 19. An element of the scaling matrix is ​​signaled in the encoded representation. The method described above.

[0334] 21. The number of elements of the scaling matrix represented in the coded representation is M and and / or the method according to Example 19, based on N.

[0335] 22. The scaling matrix shown in the coded representation for the K×L transformation matrix The first number of elements is the encoded representation for the top-left M×N coefficients without zeroing. 19. The method of claim 18, wherein the second number of elements of the scaling matrix is ​​different from the second number of elements of the scaling matrix represented by method.

[0336] 23. The scaling matrix is ​​scaled by a ratio determined from the M and / or the N. The method of Example 17 is applied by subsampling.

[0337] 24. The upper left region containing M×N coefficients is divided into a plurality of sub-regions, and each sub-region The size of the sub-region is Uw*Uh, and one element in each sub-region is signaled in the coded representation. The method of Example 23.

[0338] 25. The scaling matrix shown in the coded representation for the K×L transformation matrix The number of elements is indicated in the coded representation for the top-left M×N coefficients without zeroing. The method of Example 23, which is the same as that described in Example 23.

[0339] 26. Any of the embodiments 17 to 25, wherein K=L=64 and M=N=32. How to.

[0340] 27. Example 20, where K=L=64, M=N=32, and Uw=Uh=8. Or the method according to claim 24.

[0341] 28. When converting between a video block of a video and a coded representation of the video, A rule-based determination of whether to use a single quantization matrix based on size. wherein all image blocks having the size use the single quantization matrix. , determining and performing the transformation using the quantization matrix.

[0342] 29. The rule is that the size of the video block is, when W and H are integers, In response to the quantization matrix being smaller than W×H, only the single quantization matrix is ​​permitted. The method of Example 28.

[0343] 30. The rule may be to: in response to the width of the image block being less than a threshold, 29. The method of example 28, specifying that only one quantization matrix is ​​allowed.

[0344] 31. The rule may be: in response to the height of the image block being less than a threshold, 29. The method of example 28, specifying that only a single quantization matrix is ​​allowed.

[0345] 32. The rule is such that the single quantization matrix is ​​associated with a small size image block. 2. The second embodiment provides that the above-mentioned method does not apply to the image block having the size specified in the above embodiment. 8. The method according to claim 8.

[0346] 33. Any of the first to third embodiments, wherein a palette mode is applied to the image block. The method described above.

[0347] 34. The transforming includes encoding the video into the encoded representation. 4. A method according to any one of claims 3 to 3.

[0348] 35. The transform includes decoding the encoded representation to generate pixel values ​​for the video block. The method according to any one of Examples 1 to 33, comprising:

[0349] 36. A processing device configured to implement the method according to any one or more of the first to fifth embodiments. A video decoding device comprising:

[0350] 37. A processing device configured to implement the method according to any one or more of the first to fifth embodiments. The video encoding device includes:

[0351] 38. A computer program product having computer code stored therein, When the code is executed by a processing device, the processing device performs the processing described in any one of Examples 1 to 35. Implement the method.

[0352] The disclosed and other solutions, examples, embodiments, modules, The implementation of the functions and operations may be implemented in any manner including the structures disclosed herein and their structural equivalents. Any digital electronic circuit, including computer software, firmware or may be implemented in hardware, or in a combination of one or more thereof. The disclosed and other embodiments may include one or more computer program products. i.e. to be implemented by or to control the operation of a data processing apparatus one of computer program instructions encoded on a computer readable medium for controlling The computer-readable medium may be implemented as one or more modules. -readable storage device, machine-readable storage substrate, memory device, material providing a machine-readable propagating signal - Patents.com or a combination of one or more of these. The term may refer to, for example, a programmable processing device, a computer, or a plurality of processing devices, or any apparatus, device, or device for processing data, including a computer; This device includes a machine that, in addition to the hardware, also includes the execution environment of the computer program. The code that creates the system, e.g., processor firmware, protocol stacks, database management The components that make up the operating system, the management system, or a combination of one or more of these A propagated signal may include an artificially generated signal, e.g., a machine-generated signal. A digital signal is an electrical, optical, or electromagnetic signal that encodes information for transmission to an appropriate receiving device. It is generated to

[0353] Computer programs (programs, software, software applications) , script, or code) is a language that can be compiled or interpreted. It can be written in any form of programming language, including the Module suitable for use as a standalone program or in any computing environment. It may be deployed in any form, including as a module, component, subroutine, or other unit. A computer program does not necessarily have to be a file in a file system. A program may hold other programs or data. To a part of a file (for example, one or more scripts stored in a markup language document) It may be recorded or stored in a single file dedicated to that program. However, multiple reconciliation files (e.g., one or more modules, subprograms, or code) A computer program may be stored in a file that contains part of the program. on a single computer located at one site, or on a communications network distributed across multiple sites. Deploy it to run on multiple computers interconnected by a network. is also possible.

[0354] The processes and logic flows described herein operate on input data and produce output. execute one or more computer programs to perform functions by creating The processing and logic flow may be implemented by one or more programmable processing devices. The PLC also uses application-specific logic circuits, such as field programmable gate arrays (FPGAs). This can be done by a chip array (Chip Array) or an ASIC (Application Specific Integrated Circuit), and the device can also It may also be implemented as special purpose logic circuitry.

[0355] Suitable processing devices for the execution of a computer program include, for example, general purpose and special purpose microprocessors. Both the processor and any one or more processors of any kind of digital computer Typically, a processing unit includes a read-only memory or a random access memory. The essential elements of a computer are the A processor for executing and one or more memory devices for storing instructions and data. Generally, a computer has one or more mass storage devices for storing data. The device may include, for example, a magnetic, magneto-optical, or optical disk. to receive data from or transfer data to these mass storage devices. However, the computer may be operatively coupled to such a device. It is not necessary to have a computer suitable for storing computer program instructions and data. Computer readable media includes any form of non-volatile memory, media, and memory devices. Includes, for example, EPROM, EEPROM, flash memory devices, magnetic disks, For example, internal hard disks or removable disks, magneto-optical disks, and CD- Includes semiconductor storage devices such as ROM and DVD-ROM disks. Processing devices and memories may be supplemented by special purpose logic circuitry, or may be incorporated into the

[0356] This patent specification contains many details, which may be misleading in any respect as to the scope or scope of any subject matter or claims. The present invention should not be construed as limiting the scope of the present invention, but rather as being specific to particular embodiments of particular technologies. The description of the possible features of the present invention should be interpreted as a description of the possible features of the present invention. Certain features described in the context may be implemented in combination in one example. Conversely, various features that are described in the context of one example may also be used in multiple embodiments. Further, the features may be implemented separately or in any suitable subcombination. The compounds described above as acting in specific combinations and initially claimed as such However, one or more features from a claimed combination may, in some cases, be combined. The claimed combination may be extracted from any combination of the claimed The present invention may be directed to variations in the above combinations.

[0357] Similarly, although operations may be shown in a particular order in the figures, this is not intended to be limiting as to how the desired results are achieved. that such actions be performed in the particular order or sequence shown, in order to It should not be construed as requiring that all acts shown be performed. Also, the separation of the various system components in the examples described in this patent specification is It should not be understood that all embodiments require such separation.

[0358] Only certain implementations and examples are described and illustrated in this patent document. Other embodiments, extensions, and variations are possible based on the content provided.

Claims

1. 1. A method for processing video data, comprising the steps of: performing a first conversion between a first video block of a video and a bitstream of the video; the bitstream conforms to format rules; The format rule specifies that whether to apply a transform skip mode to the first video block is determined according to a coding condition of the first video block; the format rules specify that if a syntax element indicating availability of the transform skip mode is omitted from the bitstream, the transform skip mode is determined to be enabled because the coding conditions of the first video block indicate that a differential coding mode is used for the first video block; the transform skip mode includes skipping applying a forward transform to at least some coefficients before encoding into the bitstream, or skipping applying an inverse transform to at least some coefficients during decoding; an explicit scaling matrix of a scaling tool is disabled in response to the differential coding mode being used for the first video block; The scaling tool includes: scaling at least some coefficients representing the first video block being encoded; or descaling at least some coefficients from the bitstream being decoded.

2. 2. The method of claim 1 , wherein if the syntax element is omitted from the bitstream, the transform skip mode is determined to be disabled because the coding conditions of the first video block indicate that the differential coding mode is not used for the first video block.

3. The method of claim 1 or 2, wherein the differential coding mode comprises a differential pulse code modulation mode or a quantized residual block differential pulse code modulation mode.

4. The method of claim 1 , wherein a flat scaling matrix is ​​used in the scaling tool for the first transformation of the first image block.

5. The method of claim 4 , wherein the intermediate scaling element of the flat scaling matrix is ​​equal to 16.

6. 2. The method of claim 1 , wherein elements of the explicit scaling matrix are determined identically for a second video block coded using an intra block copy mode and a third video block coded using an inter mode.

7. 2. The method of claim 1 , wherein a second video block of a video region coded using an intra block copy mode and a third video block of the video region coded using an inter mode share the same set of explicit scaling matrices.

8. The method of claim 7 , wherein the set of explicit scaling matrices is derived from a scaling list data syntax structure of the bitstream.

9. The method of any one of claims 1 to 8, wherein the first transformation comprises encoding the first video block into the bitstream.

10. The method of any one of claims 1 to 8, wherein the first conversion comprises decoding the first video block from the bitstream.

11. 9. The method of claim 8, wherein the scaling list data syntax structure includes at least one of a first syntax element indicating whether a value of a scaling list is predicted from a reference scaling list, a second syntax element indicating the reference scaling list, a third syntax element indicating a value of a DC position of the explicit scaling matrix, or a fourth syntax element indicating a difference between a current matrix coefficient and a previous matrix coefficient.

12. The method of claim 7 , wherein the image region is a slice or a picture.

13. determining that during a second conversion between a video transform block of the video having a K×L transform block size and the video bitstream, if a secondary transform matrix is ​​applied to an S×T top-left portion of the video transform block, a scaling matrix is ​​applied to the video transform block; S, T, K, and L are integers, K is greater than or equal to S, and L is greater than or equal to T, and the scaling matrix is ​​used to scale at least some coefficients of the video transform block; The method of claim 1 , further comprising: performing the second transformation using the scaling matrix.

14. The method of claim 13 , wherein the scaling matrix is ​​effectively applied only to an S×T top-left portion of the image transform block.

15. The method of claim 13 or 14, wherein the scaling matrix comprises a default matrix or a matrix explicitly included in the bitstream.

16. The method of claim 15 , wherein the matrix elements of the default matrix are uniform and have a value equal to 16.

17. A method according to any one of claims 13 to 16, wherein in response to the image transform block of the image having the KxL transform block size containing MxN or more positions, only matrix elements within a top left MxN region of the image transform block of the image having the KxL transform block size are retained, and remaining matrix elements are set to zero, where M and N are integers.

18. 18. The method of claim 17, wherein M=32 and N=32.

19. 18. The method of claim 17, wherein if K is equal to 64, only matrix elements in the left 32 columns of the scaling matrix are retained, and if L is equal to 64, only matrix elements in the top 32 rows of the scaling matrix are retained, with remaining matrix elements being zeroed out.

20. The method of any one of claims 13 to 19, wherein the second transformation comprises encoding the video transform block into the bitstream.

21. The method of any one of claims 13 to 19, wherein the second transformation comprises decoding the video transform block from the bitstream.

22. 1. An apparatus for processing video data comprising a processor and a non-transitory memory having instructions, the instructions, when executed by the processor, causing the processor to: performing a first transformation between a first video block of a video and a bitstream of the video; the bitstream conforms to format rules; The format rule specifies that whether to apply a transform skip mode to the first video block is determined according to a coding condition of the first video block; the format rules specify that if a syntax element indicating availability of the transform skip mode is omitted from the bitstream, the transform skip mode is determined to be enabled because the coding conditions of the first video block indicate that a differential coding mode is used for the first video block; the transform skip mode includes skipping applying a forward transform to at least some coefficients before encoding into the bitstream, or skipping applying an inverse transform to at least some coefficients during decoding; an explicit scaling matrix of a scaling tool is disabled in response to the differential coding mode being used for the first video block; The scaling tool includes: scaling at least some coefficients representing the first video block being encoded; or descaling at least some coefficients from the bitstream being decoded.

23. A non-transitory computer-readable storage medium storing instructions, the instructions causing a processor to: performing a first transformation between a first video block of a video and a bitstream of the video; the bitstream conforms to format rules; The format rule specifies that whether to apply a transform skip mode to the first video block is determined according to a coding condition of the first video block; the format rules specify that if a syntax element indicating availability of the transform skip mode is omitted from the bitstream, the transform skip mode is determined to be enabled because the coding conditions of the first video block indicate that a differential coding mode is used for the first video block; the transform skip mode includes skipping applying a forward transform to at least some coefficients before encoding into the bitstream, or skipping applying an inverse transform to at least some coefficients during decoding; an explicit scaling matrix of a scaling tool is disabled in response to the differential coding mode being used for the first video block; The scaling tool includes: scaling at least some coefficients representing the first video block being encoded; or 11. A non-transitory computer-readable storage medium comprising: descaling at least some coefficients from the bitstream during decoding.

24. A method for storing a video bitstream, the method comprising: generating the bitstream of the video and storing the bitstream on a non-transitory computer-readable recording medium; the bitstream conforms to format rules; The format rule specifies that whether to apply a transform skip mode to a first video block is determined according to a coding condition of the first video block; the format rules specify that if a syntax element indicating availability of the transform skip mode is omitted from the bitstream, the transform skip mode is determined to be enabled because the coding conditions of the first video block indicate that a differential coding mode is used for the first video block; the transform skip mode includes skipping applying a forward transform to at least some coefficients before encoding into the bitstream, or skipping applying an inverse transform to at least some coefficients during decoding; an explicit scaling matrix of a scaling tool is disabled in response to the differential coding mode being used for the first video block; The scaling tool includes: scaling at least some coefficients representing the first video block being encoded; or descaling at least some coefficients from the bitstream being decoded.

Images