Method and device for video processing and medium

Abstract

The embodiment of the invention provides a scheme for video processing. A method for video processing is presented. The method comprises: for a conversion between a current video block of a video and a bitstream of the video, determining a distortion value of the current video block based on a set of distortion metrics, the set of distortion metrics comprising at least one of: a first distortion metrics determined according to a first machine learning model, a second distortion metrics determined according to a second machine learning model, and a third distortion metrics determined according to a second machine learning model; or a third distortion measure determined without using the first machine learning model and the second machine learning model; and performing a conversion based on the distortion value. In this manner, the RDO process can be improved, and thus the codec performance can be enhanced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate generally to video coding techniques, and more particularly, to distortion determination using machine learning models. Background Art

[0002] Nowadays, digital video functions are being applied to all aspects of people's lives. For video encoding / decoding, various types of video compression technologies have been proposed, such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-TH.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard. However, the encoding and decoding efficiency of traditional video encoding and decoding technologies is usually very low, which is undesirable. Summary of the invention

[0003] An embodiment of the present disclosure provides a solution for video processing.

[0004] In a first aspect, a method for video processing is provided. The method includes: for conversion between a current video block of a video and a bitstream of the video, determining a distortion value of the current video block based on a set of distortion metrics, the set of distortion metrics including at least one item: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; and performing conversion based on the distortion value.

[0005] The method according to the first aspect of the present disclosure determines a distortion value of a current video block based on a set of distortion metrics. The determined distortion value can be applied to a rate-distortion optimization (RDO) process, and thus the RDO process can be improved. Thus, codec effectiveness and codec efficiency can be improved.

[0006] In a second aspect, another method for video processing is provided. The method includes: for a conversion between a current video block of a video and a codec effect of the video, determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for the current video block; performing an RDO process on the current video block based on the determination; and performing conversion based on the RDO process.

[0007] The method according to the second aspect of the present disclosure determines whether to apply a machine learning model in the RDO process of the codec tool for the current video block. In this way, codec effectiveness and codec efficiency can be improved.

[0008] In a third aspect, a device for processing video data is provided, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method according to the first aspect or the second aspect of the present disclosure.

[0009] In a fourth aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions for causing a processor to execute the method according to the first or second aspect of the present disclosure.

[0010] In a fifth aspect, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device, wherein the method includes: determining a distortion value of a current video block of the video based on a set of distortion metrics, the set of distortion metrics including at least one of the following: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; and generating a bitstream based on the distortion value.

[0011] In a sixth aspect, a method for storing a bitstream of a video is provided. The method includes: determining a distortion value of a current video block of the video based on a set of distortion metrics, the set of distortion metrics including at least one of the following: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; and generating a bitstream based on the distortion value; and storing the bitstream in a non-transitory computer-readable recording medium.

[0012] In a seventh aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device, wherein the method includes: determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for a current video block of the video; performing the RDO process on the current video block based on the determination; and generating a bitstream based on the RDO process.

[0013] In an eighth aspect, another method for storing a bitstream of a video is provided. The method includes: determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool of a current video block of the video; performing the RDO process on the current video block based on the determination; and generating a bitstream based on the RDO process; and storing the bitstream in a non-transitory computer-readable recording medium.

[0014] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of example embodiments of the present disclosure will become more apparent from the following detailed description with reference to the accompanying drawings, in which like reference numerals generally refer to like components.

[0016] Figure 1 A block diagram showing an example video encoding and decoding system according to some embodiments of the present disclosure is shown;

[0017] Figure 2 A block diagram of a first example video encoder according to some embodiments of the present disclosure is shown;

[0018] Figure 3 A block diagram of an example video decoder according to some embodiments of the present disclosure is shown;

[0019] Figure 4 An example diagram showing an example of raster scan striping of a display picture;

[0020] Figure 5 An example diagram showing an example of rectangular strip division of a display picture;

[0021] Figure 6 An example diagram showing an example of a picture divided into tiles, bricks, and rectangular strips;

[0022] Fig. 7A An example diagram showing a CTB crossing a bottom picture border is shown;

[0023] Figure 7B An example diagram showing a CTB crossing a right picture border is shown;

[0024] Figure 7C An example diagram showing a CTB crossing the lower right picture border is shown;

[0025] Figure 8 An example diagram showing an example of an encoder block diagram is shown;

[0026] Fig. 9 An example diagram showing picture samples on an 8×8 grid and horizontal and vertical block boundaries, and an illustration of non-overlapping blocks of 8×8 samples is shown;

[0027] Fig.10An example diagram showing pixels involved in filter on / off decisions and strong / weak filter selection is shown;

[0028] Figures 11A-11D An example graph showing four 1-D (one-dimensional) directional patterns for EO sample classification is shown;

[0029] Figure 12A-12C An example diagram showing an example of a GALF filter shape is shown;

[0030] Figure 13A-13C An example diagram showing an example of relative coordinates for a 5×5 diamond filter support is shown;

[0031] Fig.14 An example diagram showing an example of relative coordinates for a 5×5 diamond filter support is shown;

[0032] Fig.15A An example diagram showing the architecture of the proposed CNN filter is shown;

[0033] Fig. 15B An example diagram showing the construction of a ResBlock (residual block) in a CNN filter is shown;

[0034] Fig.16 A flowchart of a method for video processing according to some embodiments of the present disclosure is shown;

[0035] Fig.17 Another flow chart of a method for video processing according to some embodiments of the present disclosure is shown; and

[0036] Fig.18 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0037] Throughout the drawings, same or similar reference numbers generally refer to same or similar elements. Specific embodiments

[0038] The principle of the present disclosure is now described in conjunction with some embodiments. It should be understood that these embodiments are described only for illustrative purposes to help those skilled in the art understand and implement the present disclosure, and do not constitute any limitation to the scope of the present disclosure. The disclosure described herein can be implemented in various ways except for the way described below.

[0039] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0040] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, it is considered that structures or features related to other embodiments that affect that feature are within the knowledge of those skilled in the art, whether or not explicitly described.

[0041] It should be understood that although the terms "first" and "second" etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the exemplary embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0042] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "comprise", "include", "have", "have", "include" and / or "include", when used herein, specify the presence of stated features, elements and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components and / or combinations thereof.

[0043] Example Environment

[0044] Figure 1 1 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured to generate encoded video data, and the destination device 120 may be configured to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0045] Video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system for generating video data, and / or combinations thereof.

[0046] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bitstream. The bitstream may include a bit sequence that forms a codec representation of the video data. The bitstream may include a codec picture and associated data. The codec picture is a codec representation of the picture. The associated data may include a sequence parameter set, a picture parameter set, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The encoded video data may be directly transmitted to the destination device 120 via the network 130A via the I / O interface 116. The encoded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0047] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured to be connected to an external display device interface.

[0048] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or future standards.

[0049] Figure 2 is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0050] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0051] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0052] In other examples, the video encoder 200 may include more, fewer, or different functional components. In one example, the prediction unit 202 may include an intra block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture in which the current video block is located.

[0053] Furthermore, although some components, such as the motion estimation unit 204 and the motion compensation unit 205, may be integrated, for purposes of explanation, these components are described in detail in the following sections. Figure 2 are shown separately in the example.

[0054] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0055] The mode selection unit 203 may select one of a plurality of codec modes (INTRA codec or INTER codec) based on the error result, for example, and provide the resulting intra-coded block or inter-coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra- and inter-prediction (CIIP) modes, where the prediction is based on an inter-prediction signal and an intra-prediction signal. In the case of inter-prediction, the mode selection unit 203 may also select a resolution for a motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0056] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0057] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0058] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0059] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block, and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0060] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of an adjacent video block.

[0061] In one example, motion estimation unit 204 may indicate to video decoder 300 a value in a syntax structure associated with the current video block that indicates that the current video block has the same motion information as another video block.

[0062] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0063] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0064] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0065] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0066] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0067] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0068] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0069] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0070] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0071] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives the data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy-coded data and output a bitstream including the entropy-coded data.

[0072] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0073] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0074] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0075] The entropy decoding unit 301 can retrieve the encoded bitstream. The encoded bitstream may include entropy-encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy-encoded video data, and the motion compensation unit 302 can determine motion information from the entropy-decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine the information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures of adjacent PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B strips, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from blocks adjacent in the spatial or temporal domain.

[0076] The motion compensation unit 302 may generate a motion compensated block, which may perform interpolation based on interpolation filtering. An identifier for the interpolation filter used with sub-pixel precision may be included in a syntax element.

[0077] The motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filtering used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filtering used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filtering to generate a prediction block.

[0078] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-frame coded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding and decoding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0079] The intra prediction unit 303 may use, for example, an intra prediction mode received in the bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 303 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 303 applies an inverse transform.

[0080] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 202 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block in order to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also produces the decoded video for presentation on a display device.

[0081] Some exemplary embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, rather than limiting the embodiments disclosed in the section to that section. In addition, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed technology is also applicable to other video codec technologies. In addition, although some embodiments describe the video coding and decoding steps in detail, it should be understood that the corresponding decoding steps of the cancellation codec will be implemented by the decoder. In addition, the term video processing includes video coding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates.

[0082] 1. Brief Overview

[0083] The present disclosure relates to video coding technology. Specifically, it is related to loop filters in image / video coding. It can be applied to existing video coding standards, such as High Efficiency Video Codec (HEVC), Versatile Video Codec (VVC), or standards to be finalized (e.g., AVS3). It can also be applied to future video coding standards or video codecs, or used as a post-processing method outside the encoding / decoding process.

[0084] 2. Introduction

[0085] Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. ITU-T developed H.261 and H.263, ISO / IEC developed MPEG-1 and MPEG-4 Visual, and the two organizations jointly developed the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H.262, video codec standards have been based on a hybrid video codec structure that utilizes temporal prediction plus transform codec. In order to explore future video codec technologies beyond HEVC, VCEG and MPEG jointly established the Joint Video Exploration Team (JVET) in 2015. Since then, JVET has adopted many new methods and put them into reference software called the Joint Video Exploration Model (JEM). In April 2018, the Joint Video Experts Team (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) was formed to work on the VVC standard, with the goal of reducing bitrate by 50% compared to HEVC. VVC version 1 was finalized in July 2020.

[0086] The latest version of the VVC draft, Versatile Video Codec (Draft 10), can be found at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / current_document.php?id=10399.

[0087] The latest reference software for VVC (called VTM) can be found at: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / - / tags / VTM-10.0.

[0088] 2.1 Color Space and Chroma Downsampling

[0089] A color space, also called a color model (or color system), is an abstract mathematical model that simply describes the range of colors as a tuple of numbers, usually 3 or 4 values ​​or color components (e.g., RGB). Fundamentally, a color space is an elaboration of coordinate systems and subspaces.

[0090] For video compression, the most commonly used color spaces are YCbCr and RGB. YCbCr, Y′CbCr, or Y Pb / Cb Pr / Cr, also written as YCBCR or Y'CBCR, is a family of color spaces used as part of the color image pipeline in video and digital photography systems. Y′ is the luminance component, while CB and CR are the blue-difference and red-difference chrominance components. Y′ (with a prime) is distinguished from Y, which is luminance, meaning that light intensity is non-linearly encoded based on the gamma-corrected RGB primaries.

[0091] Chroma downsampling is the practice of encoding an image by achieving a lower resolution for chroma information than for luminance information, taking advantage of the fact that the human visual system is less sensitive to color differences than to luminance. 2.1.1.4:4:4

[0093] Each of the three Y'CbCr components has the same sample rate, so there is no chroma downsampling. This scheme is sometimes used in high-end film scanners and film post-production. 2.1.2.4:2:2

[0095] The two chroma components are sampled at half the luma sample rate: the horizontal chroma resolution is halved. This reduces the bandwidth of the uncompressed video signal by one third with little visual difference. 2.1.3.4:2:0

[0097] In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is halved because the Cb and Cr channels are only sampled on every alternate line in this scheme. Therefore the data rate is the same. Cb and Cr are each downsampled by a factor of 2 both horizontally and vertically. There are three variants of the 4:2:0 scheme with different horizontal and vertical addressing.

[0098] In MPEG-2, Cb and Cr are co-located in the horizontal direction. Cb and Cr are located between pixels in the vertical direction (interval positioning).

[0099] In JPEG / JFIF, H.261, and MPEG-1, Cb and Cr are located in the gaps, in the middle of alternating luma samples.

[0100] In 4:2:0 DV, Cb and Cr are co-located horizontally. Vertically, they are co-located on alternate lines.

[0101] 2.2. Definition of video unit

[0102] A picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs covering a rectangular area of ​​a picture.

[0103] A tile is divided into one or more bricks, each brick consisting of a certain number of CTU rows within the tile.

[0104] A tile that is not divided into multiple bricks is also called a brick. However, a brick that is a proper subset of a tile is not called a tile.

[0105] A strip contains a certain number of tiles of an image or a certain number of bricks of tiles.

[0106] Two strip modes are supported, raster scan strip mode and rectangular strip mode. In raster scan strip mode, a strip contains a sequence of tiles in a tile raster scan of an image. In rectangular strip mode, a strip contains a certain number of bricks of an image that together form a rectangular region of the image. The bricks within a rectangular strip are arranged in the order of the strip's brick raster scan.

[0107] Figure 4 An example diagram 400 showing an example of raster scan striping of a display picture is shown. Figure 4 In , the image is divided into 12 tiles and 3 raster scan strips. Figure 4 A picture with 18×12 luma CTU is divided into 12 tiles and 3 raster scan strips (large amount of information).

[0108] Figure 5 An example diagram 500 showing an example of rectangular striping of a display picture is shown. Figure 5 In , the image is divided into 24 tiles (6 tile columns and 4 tile rows) and 9 rectangular strips. Figure 5 A picture with 18×12 luma CTU is divided into 24 tiles and 9 rectangular strips (large amount of information).

[0109] Figure 6 An example diagram 600 showing an example of a picture divided into tiles, bricks, and rectangular strips is shown. Figure 6 , the image is divided into 4 tiles (2 tile columns and 2 tile rows), 11 bricks (the upper left tile contains 1 brick, the upper right tile contains 5 bricks, the lower left tile contains 2 bricks, and the lower right tile contains 3 bricks), and 4 rectangular strips. Figure 6 The image in is divided into 4 tiles, 11 bricks and 4 rectangular strips (large amount of information).

[0110] 2.2.1.CTU / CTB size

[0111] In VVC, the CTU size indicated by the syntax element log2_ctu_size_minus2 in the SPS can be as small as 4×4.

[0112] 7.3.2.3 Sequence parameter set RBSP syntax

[0113]

[0114]

[0115]

[0116] log2_ctu_size_minus2 plus 2 specifies the luma codec tree block size for each CTU.

[0117] log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size.

[0118] The variables CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC are exported as follows:

[0119] CtbLog2SizeY = log2_ctu_size_minus2 + 2 (7-9)

[0120] CtbSizeY = 1 << CtbLog2SizeY (7-10)

[0121] MinCbLog2SizeY=log2_min_luma_coding_block_size_minus2 + 2 (7-11)

[0122] MinCbSizeY = 1 << MinCbLog2SizeY (7-12)

[0123] MinTbLog2SizeY = 2 (7-13)

[0124] MaxTbLog2SizeY = 6 (7-14)

[0125] MinTbSizeY = 1 << MinTbLog2SizeY (7-15)

[0126] MaxTbSizeY = 1 << MaxTbLog2SizeY (7-16)

[0127] PicWidthInCtbsY = Ceil( pic_width_in_luma_samples ÷ CtbSizeY ) (7-17)

[0128] PicHeightInCtbsY = Ceil( pic_height_in_luma_samples ÷ CtbSizeY) (7-18)

[0129] PicSizeInCtbsY = PicWidthInCtbsY * PicHeightInCtbsY (7-19)

[0130] PicWidthInMinCbsY = pic_width_in_luma_samples / MinCbSizeY (7-20)

[0131] PicHeightInMinCbsY = pic_height_in_luma_samples / MinCbSizeY (7-21)

[0132] PicSizeInMinCbsY = PicWidthInMinCbsY * PicHeightInMinCbsY (7-22)

[0133] PicSizeInSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples (7-23)

[0134] PicWidthInSamplesC = pic_width_in_luma_samples / SubWidthC (7-24)

[0135] PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC (7-25)

[0136] 2.2.2 CTUs in the Picture

[0137] Assume that the CTB / LCU size is indicated by M×N (usually M equals N, as defined in HEVC / VVC), and for a CTB located at the border of a picture (or tile or stripe or other type, taking the picture border as an example), K×L samples are within the picture border, where K < M or L < N. Fig. 7A An example diagram 700 showing a CTB passing through the bottom picture border is shown, where K = M and L < N. Figure 7B An example diagram 720 showing a CTB passing through the right picture border is shown, where K < M and L = N. Figure 7C An example diagram 740 showing a CTB passing through the bottom - right picture border is shown, where K < M and L < N. For Figure 7A-7C those CTBs depicted in

[0138] 2.3. Coding and decoding processes of typical video codecs

[0139] Figure 8 An example diagram 800 showing an example of the encoder block diagram of VVC is shown, which includes three in - loop filter blocks: Deblocking Filter (DF) 805, Sample Adaptive Offset (SAO) 806, and ALF 807. Different from DF805 that uses a predefined filter, SAO 806 and ALF 807 utilize the original samples of the current picture to reduce the mean - square error between the original samples and the reconstructed samples by adding an offset and by applying a Finite Impulse Response (FIR) filter respectively, using the coded - decoding side information representing the offset and filter coefficients. ALF 807 is located at the last processing stage of each picture and can be regarded as a tool to try to capture and repair the artifacts generated by the previous stages.

[0140] 2.4. Deblocking Filter (DB)

[0141] The input of the DB is the reconstructed samples before the in - loop filter.

[0142] First, the vertical edges in the picture are filtered. Then, the samples modified by the vertical - edge filtering process are used as the input to filter the horizontal edges in the picture. The vertical and horizontal edges in the CTB of each CTU are processed separately on the basis of the coding - decoding unit. The vertical edges of the coding - decoding blocks in the coding - decoding unit are filtered, and the filtering starts from the edge on the left - hand side of the coding - decoding block and travels through the edge in its geometric order to the right - hand side of the coding - decoding block. The horizontal edges of the coding - decoding blocks in the coding - decoding unit are filtered, and the filtering starts from the edge on the top of the coding - decoding block and travels through the edge in its geometric order to the bottom of the coding - decoding block.

[0143] Fig. 9 An example diagram 900 is shown showing picture samples on an 8x8 grid and horizontal and vertical block boundaries, as well as an illustration of non-overlapping blocks of 8x8 samples that may be deblocked in parallel.

[0144] 2.4.1. Boundary Decision

[0145] The filtering is applied to 8x8 block boundaries. Furthermore, it must be a transform block boundary or a codec subblock boundary (e.g. due to the use of affine motion prediction, ATMVP). For those boundaries that are not such boundaries, the filter is disabled.

[0146] 2.4.2. Boundary strength calculation

[0147] For the transform block boundary / codec sub-block boundary, if it is located in the 8×8 grid, it can be filtered, and the settings of bS[xDi][yDj] (where [xDi][yDj] represents the coordinates) of the edge are defined in Table 1 and Table 2 respectively.

[0148] Table 1. Boundary Strength (When SPS IBC is disabled)

[0149]

[0150] Table 2. Boundary Strength (When SPS IBC is Enabled)

[0151]

[0152]

[0153] 2.4.3. Deblocking Decisions for Luma Components

[0154] In this subsection the deblocking decision process is described. Fig.10 An example diagram 1000 showing pixels involved in filter on / off decisions and strong / weak filter selection is shown.

[0155] The wider, stronger luminance filter is used only when Condition 1, Condition 2, and Condition 3 are all TRUE.

[0156] Condition 1 is the "large block condition". This condition detects whether the samples on the P side and Q side belong to large blocks, represented by the variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. bSidePisLargeBlk and bSideQisLargeBlk are defined as follows.

[0157] bSidePisLargeBlk = ((edge ​​type is vertical and p0 belongs to a CU with width greater than or equal to 32) || (edge ​​type is horizontal and p0 belongs to a CU with height greater than or equal to 32))? TRUE: FALSE

[0158] bSideQisLargeBlk = ((edge ​​type is vertical and q0 belongs to a CU with width greater than or equal to 32) || (edge ​​type is horizontal and q0 belongs to a CU with height greater than or equal to 32))? TRUE: FALSE

[0159] Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows.

[0160] Condition 1=(bSidePisLargeBlk||bSidePisLargeBlk)? TRUE:FALSE

[0161] Next, if condition 1 is true, condition 2 will be checked further. First, the following variables are exported:

[0162] -dp0,dp3,dq0,dq3 are first derived as in HEVC

[0163] -if (side p is greater than or equal to 32)

[0164] dp0=(dp0+Abs(p50-2*p40+p30)+1)>>1

[0165] dp3=(dp3+Abs(p53-2*p43+p33)+1)>>1

[0166] -if (q side is greater than or equal to 32)

[0167] dq0=(dq0+Abs(q50-2*q40+q30)+1)>>1

[0168] dq3=(dq3+Abs(q53-2*q43+q33)+1)>>1

[0169] Condition 2 = (d < β)? TRUE: FALSE

[0170] Where d = dp0 + dq0 + dp3 + dq3.

[0171] If condition 1 and condition 2 are valid, then further check if any block uses sub-blocks:

[0172] If(bSidePisLargeBlk)

[0173]

[0174] Finally, if both conditions 1 and 2 are valid, the proposed deblocking method will check condition 3 (large block strong filter condition), which is defined as follows.

[0175] In condition 3StrongFilterCondition, export the following variables:

[0176]

[0177] For example, in HEVC, StrongFilterCondition = (dpq is less than (β>>2), sp3+sq3 is less than (3*β>>5), and Abs(p0-q0) is less than (5*t C +1)>>1)? TRUE:FALSE.

[0178] 2.4.4. Stronger deblocking filter for luminance (designed for larger blocks)

[0179] When samples on either side of the boundary belong to a large block, a bilinear filter is used. Samples belonging to a large block are defined as when width >= 32 for vertical edges and when height >= 32 for horizontal edges.

[0180] The bilinear filters are listed below.

[0181] In the above HEVC deblocking effect, the block boundary samples pi for i=0 to Sp-1 and the block boundary samples qi for j=0 to Sq-1 (pi and qi are the i-th samples in a row for vertical edge filtering, or the i-th samples in a column for horizontal edge filtering) are then replaced by linear interpolation as shown below:

[0182] -p i ′=(f i *Middles ,t +(64-f i )*P s +32)>>6), clipped to p i ±tcPD i

[0183] -q j ′=(g j *Middle s,t +(64-g j )*Q s +32)>>6), clipped to q j±tcPD j

[0184] Among them, tcPD i and tcPD j The term is the position-dependent clipping described in Section 2.4.7, and g j 、f i 、Middle s,t , P s and Q s Given below.

[0185] 2.4.5. Deblocking Control for Chroma

[0186] A chroma strong filter is used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position), and the decision meets the following three conditions: The first condition is used for boundary strength and large block decisions. The proposed filter can be applied when the block width or height orthogonal to the block edge in the chroma sample domain is equal to or greater than 8. The second and third conditions are basically the same as those used for HEVC luminance deblocking effect decisions, which are on / off decisions and strong filter decisions, respectively.

[0187] In the first decision, the boundary strength (bS) is modified for chroma filtering, and the conditions are checked sequentially. If a condition is met, the remaining conditions with lower priority are skipped.

[0188] Chroma deblocking is performed when bS is equal to 2, or when bS is equal to 1 when a large block boundary is detected.

[0189] The second and third conditions are essentially the same as the HEVC luma strong filter decision as follows.

[0190] In the second condition:

[0191] d is then derived as in HEVC luma deblocking.

[0192] When d is less than β, the second condition will be TRUE.

[0193] In the third condition, StrongFilterCondition is derived as follows:

[0194] dpq is derived as in HEVC.

[0195] sp3 = Abs(p3 - p0), as derived in HEVC.

[0196] sq3=Abs(q0-q3), as derived in HEVC.

[0197] As designed in HEVC, StrongFilterCondition = (dpq is less than (β>>2), sp3+sq3 is less than (β>>3), and Abs(p0-q0) is less than (5*t C +1)>>1).

[0198] 2.4.6. Strong deblocking filter for chroma

[0199] The following strong deblocking filters for chroma are defined:

[0200] p2′=(3*p3+2*p2+p1+p0+q0+4)>>3,

[0201] p1′=(2*p3+p2+2*p1+p0+q0+q1+4)>>3,

[0202] p0′=(p3+p2+p1+2*p0+q0+q1+q2+4)>>3.

[0203] The proposed chroma filter performs deblocking on a 4×4 chroma sample grid.

[0204] 2.4.7. Position-dependent clipping

[0205] Position dependent clipping tcPD is applied to the output samples of the luminance filtering process which involves modifying strong and long filters of 7, 5 and 3 samples at the boundaries. Assuming a quantization error distribution, it is proposed to increase the clipping value for samples that are expected to have higher quantization noise, and therefore the reconstructed sample values ​​are expected to have higher deviations from the true sample values.

[0206] For each P or Q boundary filtered with an asymmetric filter, a position-dependent threshold table is selected from the two tables provided to the decoder as side information (i.e., Tc7 and Tc3 listed in the following table) according to the result of the decision process in Section 2.4.2:

[0207] Tc7={6,5,4,3,2,1,1}; Tc3={6,4,2};

[0208] tcPD=(Sp==3)? Tc3:Tc7;

[0209] tcQD=(Sq==3)?Tc3:Tc7.

[0210] For P or Q boundaries filtered with a short symmetric filter, a lower magnitude position-dependent threshold is applied:

[0211] Tc3 = {3, 2, 1};

[0212] After defining the thresholds, the filtered p'i and q'i sample values ​​are clipped according to the tcP and tcQ clipping values:

[0213] p”i=Clip3(p’i+tcPi,p’i–tcPi,p’i);

[0214] q”j=Clip3(q’j+tcQj,q’j–tcQj,q’j);

[0215] Among them, p' i and q' i is the filtered sample value, p" i and q” j is the clipped output sample value, and tcP i tcP i is the clipping threshold derived from the VVC tc parameters as well as tcPD and tcQD. Function Clip3 is the clipping function specified in VVC.

[0216] 2.4.8. Sub-block deblocking adjustment

[0217] To enable parallel friendly deblocking and sub-block deblocking using long filters, the long filter is restricted to modifying a maximum of 5 samples on one side using sub-block deblocking (affine or ATMVP or DMVR), as shown in the luma control for long filters. Additionally, sub-block deblocking is adjusted so that sub-block boundaries on the 8×8 grid close to CU or implicit TU boundaries are restricted to modifying a maximum of two samples on each side.

[0218] The following applies to sub-block boundaries that are not aligned with CU boundaries.

[0219]

[0220]

[0221] Where an edge equal to 0 corresponds to a CU boundary, an edge equal to 2 or equal to orthogonalLength-2 corresponds to a sub-block boundary 8 samples from a CU boundary, etc. Where implicitTU is true if implicit splitting of TUs is used.

[0222] 2.5.SAO

[0223] The input of SAO is the reconstructed samples after DB. The concept of SAO is to reduce the average sample distortion of a region by first classifying the region samples into multiple categories with the selected classifier, obtaining an offset for each category, and then adding the offset to each sample of the category, where the classifier index and the offset of the region are encoded and decoded in the bitstream. In HEVC and VVC, the region (the unit for SAO parameter signaling) is defined as CTU.

[0224] Two SAO types are used in HEVC to meet the low complexity requirements. These two types are edge offset (EO) and band offset (BO), which will be discussed in further detail below. The index of the SAO type is encoded (it is in the range of [0, 2]). For EO, ​​sample classification is based on the comparison between the current sample and the neighboring samples according to the 1-D directional pattern (horizontal, vertical, 135° diagonal and 45° diagonal).

[0225] Fig.11A An example graph 1100 showing 1-D directional patterns for classification of EO samples with horizontal (EO class=0) is shown. Fig. 11B An example graph 1120 showing 1-D directional patterns for EO sample classification with vertical (EO class=1) is shown. Fig. 11C An example graph 1140 showing 1-D directional patterns for classification of EO samples having a 135° subtending angle (EO class = 2) is shown. Fig.11D An example graph 1160 showing 1-D directional patterns for classification of EO samples having a 45° subtending angle (EO class = 3) is shown.

[0226] For a given EO class, each sample in the CTB is classified into one of five categories. The current sample value, labeled "c", is compared with its two neighboring sample values ​​along the selected 1-D pattern. Table 1 summarizes the classification rules for each sample. Categories 1 and 4 are associated with local troughs and local peaks along the selected 1-D pattern, respectively. Categories 2 and 3 are associated with concave corners and convex corners along the selected 1-D pattern, respectively. If the current sample does not belong to EO category 1-4, it is category 0 and SAO is not applied.

[0227] Table 3: Sample classification rules for edge offset

[0228] category condition 1 c < a and c < b 2 (c<a&&c==b)||(c==a&&c<b) 3 (c>a&&c==b)||(c==a&&c>b) 4 c>a&&c>b 5 None of the above

[0229] 2.6. Adaptive Loop Filter Based on Geometric Transformation in JEM

[0230] The input of DB is the reconstructed samples after DB and SAO. The sample classification and filtering process is based on the reconstructed samples after DB and SAO.

[0231] In JEM, a geometric transform based adaptive loop filter (GALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 2×2 block based on the direction and activity of the local gradient.

[0232] Filter shape

[0233] Fig. 12A An example diagram 1200 showing an example of a GALF filter shape having a 5×5 diamond is shown. Fig. 12B An example diagram 1220 showing an example of a GALF filter shape having a 7×7 diamond is shown. Fig. 12C An example diagram 1240 showing an example of a GALF filter shape having a 9×9 diamond is shown.

[0234] In JEM, up to three diamond filter shapes can be selected for the luminance component (e.g. Figure 12A-12C ). An index is signaled at the picture level to indicate the filter shape used for the luma component. Each square represents a sample, and Ci (i is 0 to 6 (left), 0 to 12 (center), 0 to 20 (right)) represents the coefficient to be applied to that sample. For the chroma components in a picture, a 5×5 diamond shape is always used.

[0235] 2.6.1.1. Block classification

[0236] Each 2×2 block is classified into one of 25 classes. The classification index C is based on its directionality D and activity The quantized values ​​are derived as follows:

[0237]

[0238] In order to calculate D and First, use the 1-D Laplacian operator to calculate the gradient in the horizontal, vertical and two diagonal directions:

[0239]

[0240]

[0241]

[0242]

[0243] The indices i and j refer to the coordinates of the top left sample in the 2×2 block, and R(i,j) indicates the reconstructed sample at coordinates (i,j).

[0244] Then, the maximum and minimum values ​​of D for the horizontal and vertical gradients are set to:

[0245]

[0246] And, the maximum and minimum values ​​of the gradients in the two diagonal directions are set to:

[0247]

[0248] To derive the value of the directivity D, these values ​​are compared with each other and with two thresholds t1 and t2:

[0249] Step 1. If and If both are true, D is set to 0.

[0250] Step 2. If Then continue from step 3; otherwise, continue from step 4.

[0251] Step 3. If Then D is set to 2; otherwise D is set to 1.

[0252] Step 4. If Then D is set to 4; otherwise D is set to 3.

[0253] The activity value A is calculated as:

[0254]

[0255] A is further quantized to a range of 0 to 4 (inclusive), and the quantized value is represented as

[0256] For the two chroma components in a picture, no classification method is applied, ie, a single set of ALF coefficients is applied for each chroma component.

[0257] 2.6.1.2. Geometric transformation of filter coefficients

[0258] Fig.13A An example graph 1300 showing relative coordinates for a 5x5 diamond filter support (diagonally) is shown. Fig. 13B An example graph 1320 showing relative coordinates for a 5x5 diamond filter support (flipped vertically) is shown. Fig. 13C An example graph 1340 showing relative coordinates for a 5x5 diamond filter support (rotation) is shown.

[0259] Before filtering each 2×2 block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients f(k,l) associated with the coordinates (k,l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks where the ALF is applied more similar by aligning their directionality.

[0260] Three geometric transformations are introduced, including diagonal, vertical flipping and rotation:

[0261] Diagonal: f D (k,l)=f(l,k),

[0262] Flip vertically: f V (k,l)=f(k,Kl-1), (9)

[0263] Rotation: f R (k,l)=f(Kl-1,k).

[0264] Where K is the size of the filter and 0≤k,l≤K-1 are the coefficient coordinates such that position (0,0) is located at the upper left corner and position (K-1,K-1) is located at the lower right corner. The transform is applied to the filter coefficients f(k,l) according to the gradient value calculated for the block. The relationship between the transform and the four gradients in the four directions is summarized in Table 4. Figure 12A-12C The transform coefficients for each position based on a 5×5 diamond are shown.

[0265] Table 4 shows the mapping of gradients and transformations calculated for a block.

[0266] Gradient Value Transform gd2 < gd1 and gh < gv No transformation gd2 < gd1 and gv < gh Diagonal gd1 < gd2 and gh < gv Flip Vertically gd1 < gd2 and gv < gh Rotation

[0267] 2.6.1.3. Filter parameter signaling

[0268] In JEM, GALF filter parameters are signaled for the first CTU, i.e., after the slice header and before the SAO parameters of the first CTU. Up to 25 sets of luminance filter coefficients can be signaled. In order to reduce bit overhead, filter coefficients of different classifications can be merged. Moreover, the GALF coefficients of the reference picture are stored and allowed to be reused as the GALF coefficients of the current picture. The current picture can choose to use the GALF coefficients stored for the reference picture and bypass the GALF coefficient signaling. In this case, the index is signaled only to one of the reference pictures, and for the current picture, the stored GALF coefficients of the indicated reference picture are inherited.

[0269] To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. At the beginning of decoding a new sequence, the candidate list is empty. After decoding a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (i.e., 6 in the current JEM), the new filter set overwrites the oldest set in decoding order, and that is, the first-in-first-out (FIFO) rule is applied to update the candidate list. In order to avoid duplication, sets can only be added to the list when the corresponding picture does not use GALF temporal prediction. In order to support temporal scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a temporal layer. More specifically, each array allocated by the temporal layer index (TempIdx) can constitute a filter set of a previously decoded picture with a lower TempIdx. For example, the kth array is allocated to be associated with a TempIdx equal to k, and it only contains filter sets from pictures with a TempIdx less than or equal to k. After encoding and decoding a picture, the filter set associated with the picture will be used to update those arrays associated with equal to or greater than TempIdx.

[0270] Temporal prediction of the GALF coefficients is used for inter-coded frames to minimize signaling overhead. For intra-frames, temporal prediction is not available and 16 fixed filter sets are assigned to each class. To indicate the use of fixed filters, a flag for each class is signaled and, if necessary, the index of the selected fixed filter is signaled. Even when a fixed filter is selected for a given class, the coefficients of the adaptive filter f(k, l) can still be sent for that class, in which case the coefficients of the filter to be applied to the reconstructed image are the sum of the two coefficient sets.

[0271] The filtering process of the luma component can be controlled at the CU level. A flag is signaled to indicate whether GALF is applied to the luma component of the CU. For chroma components, whether GALF is applied is indicated only at the picture level.

[0272] 2.6.1.4. Filtering process

[0273] At the decoder side, when GALF is enabled for a block, each sample R(i,j) within the block is filtered to obtain the sample value R′(i,j) as shown below, where L represents the filter length and f m,n represents the filter coefficient, and f(k,l) represents the decoded filter coefficient.

[0274]

[0275] Fig.14An example diagram 1400 showing an example of relative coordinates for a 5x5 diamond filter support is shown. Fig.14 An example of relative coordinates for a 5×5 diamond filter support is shown, assuming that the coordinates (i, j) of the current sample are (0, 0). Samples in different coordinates filled with the same color are multiplied by the same filter coefficients.

[0276] 2.7. Geometric Adaptive Loop Filter (GALF) in VVC

[0277] GALF in VTM-4

[0278] In VTM4.0, the filtering process of the adaptive loop filter is performed as follows:

[0279] O(x,y)=∑ (i,j) w(i,j).I(x+i,y+j), (11)

[0280] Among them, sample I(x+i,y+j) is the input sample, O(x,y) is the filtered output sample (ie, the filter result), and w(i,j) represents the filter coefficient. In practice, in VTM4.0, integer operations are used to implement fixed-point precision calculations:

[0281]

[0282] Where L represents the filter length and w(i,j) is the filter coefficient under fixed-point precision.

[0283] The current design of GALF in VVC has the following major changes compared to the current design of GALF in JEM:

[0284] 1) Adaptive filter shapes are removed. For luma components, only 7×7 filter shapes are allowed, and for chroma components, only 5×5 filter shapes are allowed.

[0285] 2) Signaling of ALF parameters is removed from slice / picture level to CTU level.

[0286] 3) The calculation of the class index is performed in the 4×4 level instead of the 2×2 level. In addition, the downsampled Laplacian calculation method is also used for ALF classification. More specifically, there is no need to calculate the horizontal / vertical / 45 diagonal / 135 degree gradient for each sample within a block. Instead, 1:2 downsampling is used.

[0287] 2.8. Nonlinear ALF in Current VCC

[0288] 2.8.1. Filter reformulation

[0289] Without affecting the coding efficiency, equation (11) can be restated as follows:

[0290] O(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j).(I(x+i,y+j)-I(x,y)), (13)

[0291] where w(i,j) are the same filter coefficients as in equation (11) [except that w(0,0) is equal to 1 in equation (13) and 1-∑ (i,j)≠(0,0) w(i,j)].

[0292] Using the above filter formula of (13), VVC introduces nonlinearity to make the ALF more effective by using a simple clipping function to reduce the influence of neighboring sample values ​​(I(x+i,y+j)) when they differ too much from the current sample value (I(,y)) being filtered.

[0293] More specifically, the ALF filter is modified as follows:

[0294] O′(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j).K(I(x+i,y+j)-I(x,y),k(i,j)), (14)

[0295] where K(d,b)=min(b,max(-b,d)) is the clipping function and k(i,j) is the clipping parameter, which depends on the (i,j) filter coefficients. The encoder performs an optimization to find the best k(i,j).

[0296] In some implementations, a clipping parameter k(i,j) is specified for each ALF filter, and one clipping value is signaled for each filter coefficient. This means that, in the bitstream for each luma filter, up to 12 clipping values ​​may be signaled, and for chroma filters, up to 6 clipping values ​​may be signaled.

[0297] In order to limit the signaling cost and encoder complexity, only 4 fixed values ​​are used which are the same for slices INTER (inter-frame) and INTRA (intra-frame).

[0298] Because the variance of local differences for luma is usually higher than that for chroma, two different sets of luma and chroma filters are applied. A maximum sample value in each set (here 1024 for a bit depth of 10 bits) is also introduced so that clipping can be disabled if not needed.

[0299] The set of clipping values ​​is provided in Table 5. The 4 values ​​have been chosen by splitting the entire range of sample values ​​for luma (encoded in 10 bits) and the range from 4 to 1024 for chroma approximately equally in the logarithmic domain.

[0300] More precisely, the brightness table of the clipped values ​​has been obtained by the following formula:

[0301]

[0302] Similarly, the chromaticity table for the clipped value is obtained according to the following formula:

[0303]

[0304] Table 5 Authorized clipping values

[0305]

[0306] The selected clipping value is encoded and decoded in the "alf_data" syntax element by using the Golomb coding scheme corresponding to the index of the clipping value in the above Table 5. This coding scheme is the same as that used for the filter index.

[0307] 2.9. Convolutional Neural Network-Based Loop Filters for Video Codecs

[0308] Convolutional Neural Networks

[0309] In deep learning, convolutional neural networks (CNN or ConvNet) are a type of deep neural network that is most commonly used to analyze visual images. They have been very successfully applied in image and video recognition / processing, recommendation systems, image classification, medical image analysis, and natural language processing.

[0310] CNNs are regularized versions of multilayer perceptrons. Multilayer perceptrons usually imply fully connected networks, that is, each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks makes them prone to overfitting the data. Typical regularization involves adding some form of magnitude measure of the weights to the loss function. CNNs take a different approach to regularization: they exploit hierarchical patterns in the data and use smaller and simpler patterns to assemble more complex patterns. Therefore, on the scale of connectivity and complexity, CNNs are at the lower extreme.

[0311] Compared to other image classification / processing algorithms, CNN uses relatively little preprocessing. This means that the network learns filters that are hand-designed in traditional algorithms. Independence from prior knowledge and human effort in feature design is a major advantage.

[0312] 2.9.2. Deep Learning for Image / Video Encoding and Decoding

[0313] Image / video compression based on deep learning usually has two meanings: end-to-end compression based solely on neural networks and traditional frameworks enhanced by neural networks. The first type usually adopts an autoencoder-like structure, implemented by convolutional neural networks or recurrent neural networks. Although relying solely on neural networks for image / video compression can avoid any manual optimization or manual design, the compression efficiency may not be satisfactory. Therefore, the work distributed in the second type uses neural networks as an auxiliary and enhances the traditional compression framework by replacing or enhancing some modules. In this way, they can inherit the advantages of highly optimized traditional frameworks. For example, a fully connected network for intra-frame prediction is proposed. In addition to intra-frame prediction, deep learning has also been developed to enhance other modules. For example, the in-loop filter of HEVC is replaced by a convolutional neural network, and promising results are achieved. Neural networks are applied to improve the arithmetic codec engine.

[0314] 2.9.3. In-loop filtering based on convolutional neural networks

[0315] In lossy image / video compression, the reconstructed frame is an approximation of the original frame because the quantization process is irreversible and thus causes distortion in the reconstructed frame. To mitigate this distortion, a convolutional neural network can be trained to learn the mapping from the distorted frame to the original frame. In practice, training must be performed before deploying CNN-based in-loop filtering.

[0316] 2.9.3.1. Training

[0317] The goal of the training process is to find optimal values ​​for the parameters including weights and biases.

[0318] First, a codec (eg, HM, JEM, VTM, etc.) is used to compress the training dataset to generate distorted reconstructed frames.

[0319] The reconstructed frame is then fed into the CNN, and the cost is calculated using the output of the CNN and the ground truth frame (original frame). Common cost functions include SAD (sum of absolute differences) and MSE (mean square error). Next, the gradient of the cost with respect to each parameter is derived through the back propagation algorithm. Using the gradient, the value of the parameter can be updated. The above process is repeated until the convergence criterion is met. After completing the training, the derived optimal parameters are saved for the inference phase.

[0320] 2.9.3.2. Convolution process

[0321] During convolution, the filter moves across the image from left to right and top to bottom, changing one pixel column in the horizontal move and one pixel row in the vertical move. The amount of movement between applying the filter to the input image is called the stride, and it is almost always symmetric in the height and width dimensions. For both height and width movement, one or more of the default strides in two dimensions are (1, 1).

[0322] Fig.15A An example diagram 1500 showing the architecture of the proposed CNN filter is shown. Fig. 15B An example diagram 1550 showing the construction of a ResBlock (residual block) in a CNN filter is shown. In most deep convolutional neural networks, residual blocks are used as basic modules and are stacked several times to construct the final network, where in one example, the residual block is obtained by combining a convolutional layer, a ReLU / PreLU activation function, and a convolutional layer, as shown in FIG. Fig. 15B as shown in .

[0323] 2.9.3.3. Reasoning

[0324] In the inference phase, the distorted reconstructed frames are fed into the CNN and processed by the CNN model whose parameters have been determined in the training phase. The input samples of the CNN can be the reconstructed samples before or after DB, or the reconstructed samples before or after SAO, or the reconstructed samples before or after ALF.

[0325] 3. Question

[0326] Current NN filters have the following problems:

[0327] 1. The prior art design of the NN filter is applied only after reconstruction of all blocks before the in-loop filtering process within a slice. Therefore, the loss reduction due to the NN filter is not taken into account during the rate-distortion optimization (RDO) processes such as intra mode selection, partition selection, intra mode selection, inter mode selection, transform kernel selection, etc.

[0328] The encoding and decoding performance is suboptimal considering the following factors:

[0329] a. The best mode for the current block selected during the RDO process (e.g., codec method / partition size)

[0330] can be wrong because the distortion is calculated without applying the NN filter.

[0331] b. The reconstruction of the current block and the associated coded information have a great impact on the coding of subsequent blocks (eg, due to intra prediction or motion prediction). If the current block does not select the best mode, the coding performance of the subsequence blocks will also be suboptimal.

[0332] 4. Detailed plan

[0333] The following detailed embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow way. Furthermore, these embodiments can be combined in any way.

[0334] In order to solve the above problems, it is proposed to consider NN filters based on video content during the rate-distortion optimization (RDO) process. This disclosure explains how to use NN filter models to extend RDO authority, how to use NN filter models to select modes (e.g., intra-frame mode, partition mode, inter-frame mode or transform core), and how to control the use of NN filter models.

[0335] In the present disclosure, the NN filter may be any type of NN filter, such as a convolutional neural network (CNN) filter; alternatively, it may also be applied to non-NN based filters. In the following discussion, the NN filter may also be referred to as a CNN filter.

[0336] In the following discussion, a video unit may be a sequence, a picture, a slice, a tile, a brick, a sub-picture, a CTU / CTB, a CTU / CTB row, one or more CU / CBs, one or more CTU / CTBs, one or more VPDUs (virtual pipeline data units), a sub-region within a picture / slice / tile / brick. A father video unit refers to a unit that is larger than a video unit. Typically, a father unit will contain several video units. For example, when a video unit is a CTU, a parent unit may be a slice, a CTU row, multiple CTUs, etc.

[0337] The width and height of the video unit are denoted as W and H respectively.

[0338] On Mode Selection in RDO Process Using Neural Network Filter Model

[0339] 1. Whether and / or how to utilize the NN filter model (or calculate the rate-distortion cost) during the RDO process may depend on the distortion D without the NN filter model. ORG and / or the distortion D with the nth NN filter model (model index is n-1, where n>=1) n NNLF The RDO criterion is recorded as J = D + lambda * R. The distortion D with the nth NN filter n NNLF " may mean that the reconstructed samples are filtered by the nth NN filter, and the filtered reconstructed samples are compared with the original samples to derive the distortion.

[0340] a. In one example, D in the RDO criterion is the distortion with the nth NN filter model. D = D n NNLF

[0341] i. In one example, n is 1, 2, or 3.

[0342] b. In one example, D in the RDO criterion is the minimum value of the distortion without the NN filter model and the distortion with the nth NN filter model. D = min(D n NNLF ,D ORG ).

[0343] i. In one example, n is 1, 2, or 3.

[0344] c. In one example, D in the RDO criterion is the distortion of the best one of the NN filter models.

[0345] i. In one example, the best NN filter model is selected by distortion.

[0346] ii. In one example, the best NN filter model is the default model.

[0347] d. In one example, D in the RDO criterion is the minimum of the distortion without the NN filter model and the distortion with the best one of the NN filter models.

[0348] i. In one example, the best NN filter model is selected by distortion.

[0349] ii. In one example, the best NN filter model is the default model.

[0350] e. In one example, D in the RDO criterion is a scaled version without the distortion of the NN filter model, i.e., D = f*D ORG

[0351] i. In one example, f is 1.0, 0.9 or 1.1.

[0352] f. In one example, D in the RDO criterion is derived based on the distortion without the NN filter model and the distortion with the nth NN filter model.

[0353] i. In one example, D = f0*D ORG+ f1*D n NNLF .

[0354] 1) In one example, f0 = 0.0, f1 = 1.0

[0355] 2) In one example, f0=1.0, f1=0.0

[0356] ii. In one example, D = min(f0*D ORG ,f1*D n NNLF ).

[0357] 1) In one example, f0=1.0, f1=1.0

[0358] g. In one example, D in the RDO criterion can be a combination of multiple calculation methods in the above invention.

[0359] i. In one example, the combination of calculation methods may depend on the codec statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.).

[0360] 1) In one example, the combination of calculation methods may depend on the type of candidate pattern.

[0361] 2) In one example, when all or one of the T candidate modes are not partition modes, D in the RDO criterion is a combination of distortion with the pth NN filter model and / or distortion without the NN filter model.

[0362] a. In one example, when the pth NN filter model is not applied to one or all of the T candidate modes,

[0363] The D in the RDO criterion is the distortion without the NN filter model (D = D ORG ).

[0364] b. In one example, when the pth NN filter model is applied to one or all of the T candidate patterns, RDO

[0365] The D in the criterion is the distortion with the p-th NN filter model (D = D p NNLF ).

[0366] 3) In one example, when all or one of the T candidate modes are partition modes, D in the RDO criterion is a combination of distortion with the qth NN filter model and / or distortion with the pth NN filter model and / or distortion without a NN filter model.

[0367] ii. In one example, the combination may depend on the use of the qth filter model and / or the use of the pth filter model.

[0368] 1) In one example, when the qth NN filter model is applied to one or all of the T candidate modes, D in the RDO criterion is the minimum distortion value (D=min) between the distortion without the NN filter model and the distortion with the qth filter model.

[0369] (D q NNLF , D ORG ))

[0370] 2) In one example, when the qth NN filter model is not applied to one or all of the T candidate modes, D in the RDO criterion is a combination of the distortion with the pth NN filter model and the distortion without the NN filter model.

[0371] a. In one example, when the pth NN filter is applied to all candidate modes among the T candidate modes, D in the RDO criterion is

[0372] The distortion of the filter model (D = D p NNLF ).

[0373] b. In one example, when the pth NN filter model is not applied to one or all of the T candidate modes, D in the RDO criterion is no NN

[0374] The distortion of the filter model (D = D ORG ).

[0375] iii. In one example, the method of calculating D in the RDO criterion can be applied according to a specific or adaptive order.

[0376] 1) In one example, the order of applying the methods for calculating D in the RDO criterion may depend on the codec mode / codec statistics (eg, prediction mode, qp, temporal layer, slice type, etc.) of the video unit.

[0377] 2) In one example, (D = min (D q NNLF , D ORG ))

[0378] can have a ratio D = D p NNLF Higher priority.

[0379] 3) In one example, D = D p NNLF can have a ratio D = D ORG

[0380] Higher priority.

[0381] iv. In the above invention, the pth NN filter model may be a deblocking filter or SAO or ALF.

[0382] v. In the above invention, the qth NN filter model can be a CNN filter.

[0383] vi. In the above invention, the pth NN filter model can be the same as the qth NN filter model.

[0384] 1) In one example, p may be equal to q.

[0385] vii. In the above invention, the T candidate modes may be modes equivalent to the RDO criterion denoted as J=D+lambda*R.

[0386] 1) In one example, the number T of candidate patterns may be equal to 1 or 2 or 3 or 4.

[0387] h. In the above invention, the parameter f or f0 or f1 can be set according to the time domain layer and / or slice type.

[0388] i. In the above invention, the parameter f or f0 or f1 can be set according to QP (Quantization Parameter).

[0389] j. In the above invention, the parameter f or f0 or f1 can be set according to the configuration (for example, full intra-frame, random access, low delay B, low delay P, etc.).

[0390] k. In one example, the same usage rules of NN filtering should be applied to calculate the RD costs of all candidate modes and / or partitioning methods for a block.

[0391] 2. Whether and / or how to utilize the NN filter model (or calculate the rate-distortion cost) during the RDO process may depend on the distortion D without the NN filter model OR G and / or a combination of distortions with multiple NN filter models. The RDO criterion is recorded as J=D+lambda*R.

[0392] a. In one example, D in the RDO criterion is the minimum value of distortion with m NN filter models.

[0393] i. In one example, m is the available number of constructed NN filter models.

[0394] ii. In one example, m depends on the type of video unit.

[0395] iii. In one example, m depends on the signaled signal parameters of the video unit.

[0396] iv. In one example, m is the default value.

[0397] v. In one example, m is 0, 1, 2, 3, 4, etc.

[0398] b. In one example, D in the RDO criterion is the minimum of the distortion without the NN filter model and the distortion with m NN filter models.

[0399] i. In one example, m is the available number of constructed NN filter models.

[0400] ii. In one example, m depends on the type of video unit.

[0401] iii. In one example, m depends on signaled parameters of the video unit.

[0402] iv. In one example, m is the default value.

[0403] v. In one example, m is 0, 1, 2, 3, 4, etc.

[0404] c. In one example, D in the RDO criterion is a combination of the distortion without the NN filter model and the distortion with m NN filter models.

[0405] i. In one example, D = f0*D ORG+ ∑f n *D n NNLF .

[0406] d. In the above invention, the parameter m or f0 or f n Can be set according to time domain layer and / or slice type.

[0407] e. In the above invention, the parameter m or f0 or f n Can be set according to QP (Quantization Parameter).

[0408] f. In one example, the same rules of use of NN filtering should be applied to calculate the RD costs of all candidate modes and / or partitioning methods for a block.

[0409] 3. Distortion D without NN filter model ORG Can depend on other filters LF different from the NN filter model (e.g., deblocking, ALF) i . With LF i Distortion D i LF It can mean reconstructing samples through LF i The filter is filtered and the filtered reconstructed samples are compared with the original samples to derive the distortion.

[0410] a. In one example, before the NN filtering model, LF i Can be applied to reconstruct samples.

[0411] b. In one example, the NN filter model can be used in LF i Afterwards it is applied.

[0412] c. In one example, D ORG is a filter with LF i Distortion D i LF A scaled version of D ORG =f*D LF .

[0413] d. In one example, D ORG is a combination of distortion and multiple other filters, i.e., D ORG =∑f i *D i LF .

[0414] About the implementation of NN filter model in RDO process

[0415] 4. The input to the NN filter model in the RDO process may include signals from the current video block and / or neighboring blocks.

[0416] a. In one example, the prediction signal and / or partition information and / or reconstruction signal from the current block can participate in the NN filter process.

[0417] b. In one example, other information from one / multiple reference frames can participate in the NN filtering process of the current block.

[0418] i. In one example, the co-located blocks from the first frame in List-0 and / or the co-located blocks from the first frame in List-1 may participate in the NN filter process.

[0419] ii. In one example, one or more motion compensated reference blocks may participate in NN

[0420] Filter process.

[0421] c. In one example, prediction signals and / or partition information and / or pixels from at least one neighboring block may participate in the NN filter process.

[0422] i. In one example, the neighboring block may be located to the left, above, or to the upper left of the current block.

[0423] ii. In one example, samples of neighboring blocks may be reconstructed.

[0424] iii. In one example, the neighboring block may be located on the right side or below the current block.

[0425] iv. In one example, samples of neighboring blocks may not be reconstructed.

[0426] v. In one example, the samples of the neighboring blocks may be the original signal.

[0427] vi. In one example, samples of neighboring blocks can be derived from the current block.

[0428] d. In one example, the above inventions can be combined in any manner.

[0429] About the use of NN filter model in RDO process

[0430] 5. Whether and / or how to utilize the NN filter model in the RDO process may depend on the encoding and decoding statistics of the video unit (e.g., prediction mode, qp, temporal layer, slice type, etc.).

[0431] a. Whether and / or how to utilize the NN filter model in the RDO process may depend on the size of the video unit.

[0432] i. In one example, when W<=T1 and / or H<=T2 (eg, T1=T2=64), the NN filter model can be utilized in the RDO process.

[0433] ii. In one example, when W>=T1 and / or H>=T2 (e.g.,

[0434] When T1=T2=16), the NN filter model can be used in the RDO process.

[0435] iii. In one example, when W*H>=T1 (eg, T1=64), NN

[0436] The filter model can be utilized in the RDO process.

[0437] iv. In one example, when W*H<=T1 (eg, T1=4096),

[0438] The NN filter model can be utilized in the RDO process.

[0439] b. Whether and / or how to utilize the NN filter model in the RDO process may depend on the color component

[0440] i. In one example, the NN filter model is applied only in component X.

[0441] 1) In one example, X is luminance (Y) or chrominance (Cb or Cr).

[0442] ii. In one example, the NN filter model is applied to all color components.

[0443] c. Whether and / or how to utilize the NN filter model in the RDO process may depend on the rate-distortion cost without the NN filter model. A is the cost of mode A without the NN filter model, and J B is the cost of mode B without the NN filter model.

[0444] i. In one example, when the cost J A Greater than and / or equal to cost J B Multiples of (J A >f0*J B or J A > = f0*J B ), the NN filter model may not be applied.

[0445] 1) In one example, f0 is 1.0, 1.001, 1.005, 1.05, 1.01.

[0446] ii. In one example, when the cost J B Greater than and / or equal to cost J A Multiples of (J B >f1*J A or J B > = f1*J A ), the NN filter model may not be applied.

[0447] 1) In one example, f1 is 1.0, 1.001, 1.005, 1.05, 1.01.

[0448] iii. In one example, J A >f0*J B ||J B >f1*J A .

[0449] iv. In one example, when cost J A With cost J B The ratio between them is greater than and / or equal to the threshold value (J A / J B >f2 or J A / J B >=f2), the NN filter model may not be applied.

[0450] 1) In an example, f2 is 1.0, 1.001, 1.005, 1.05, 1.01.

[0451] v. In an example, when the ratio between cost J A and cost J B is less than and / or equal to a threshold (J A / J B <f3 or J A / J B < = f3), the NN filter model may not be applied.

[0452] 1) In an example, f3 is 1.0, 1.001, 1.005, 1.05, 1.01.

[0453] vi. In an example, J A / J B > f2 || J A / J B <f3

[0454] vii. In an example, when cost J A is equal to or / and less than or / and greater than a threshold Th A and / or cost J B is equal to or / and less than or / and greater than a threshold Th B the NN filter model may not be applied or may be applied.

[0455] 1) In an example, Th A or Th B is equal to MAX_DOUBLE as the default value.

[0456] 2) In an example, Th A or Th B is equal to 1.7e+308.

[0457] viii. In an example, the NN filters for mode A and mode B can be applied according to a specific or adaptive order.

[0458] 1) In an example, the order of applying the NN filters to mode A and mode B can depend on the coding / decoding mode / coding / decoding statistics of the video unit (e.g., prediction mode, qp, temporal layer, slice type, etc.).

[0459] 2) In an example, the order of applying the NN filters to mode A and mode B can depend on J A and J B .

[0460] a. In an example, when J A is greater than and / or equal to JB When applying the NN filter to mode A, it can have a better effect than applying the NN

[0461] Filters are applied in mode B with higher priority.

[0462] b. In one example, when J B Greater than and / or equal to J A When applying the NN filter to mode B, it can have a better effect than applying the NN

[0463] Filters are applied in mode A with higher priority.

[0464] ix. In one example, J A It can be equal to D+lambda*R.

[0465] 1) In one example, D may be equal to D ORG .

[0466] x. In one example, J B It can be equal to D+lambda*R.

[0467] 1) In one example, D may be equal to D ORG .

[0468] xi. In one example, the above inventions can be combined in any manner.

[0469] xii. In the above invention, the parameter Th A ,Th B , f0, f1, f2 or f3 can be set according to the time domain layer.

[0470] xiii. In the above invention, the parameter Th A ,Th B , f0, f1, f2 or f3 can be set according to QP (quantization parameter)

[0471] xiv. In the above invention, the parameter Th A ,Th B , f0, f1, f2 or f3 can be set according to the stripe type.

[0472] xv. In the above invention, the parameter Th A ,Th B , f0, f1, f2 or f3 can be set according to the configuration (e.g., full intra-frame, random access, low delay B, low delay P, etc.).

[0473] d. Whether and / or how to utilize the NN filter model in the RDO process may depend on the rate-distortion cost without the NN filter model and the rate-distortion cost with the NN filter model. Ais the cost of mode A without the NN filter model, and J' B is the cost of mode B with NN filter model.

[0474] i. In one example, whether to apply the NN filter to mode A may depend on the cost J A With cost J' B .

[0475] ii. In one example, J A It can be equal to D+lambda*R.

[0476] 1) In one example, D may be equal to D ORG .

[0477] iii. In one example, J' B It can be equal to D+lambda*R.

[0478] 1) In one example, D can be equal to min(D n NNLF , D ORG ).

[0479] 2) In one example, D may be equal to D n NNLF .

[0480] iv. In one example, when cost J A Greater than and / or equal to cost J' B Multiples of (J A >f0*J' B or J A > = f0*J' B ), the NN filter model may not be applied to mode A.

[0481] vIn one example, when the cost J' B Greater than and / or equal to cost J A Multiples of (J' B >f1*J A or J' B > = f1*J A ), the NN filter model may not be applied to mode A.

[0482] vi. In one example, J A >f0*J' B ||J' B >f1*J A .

[0483] vii. In one example, when the cost J A With cost J' BThe ratio between is greater than and / or equal to the threshold value (J A / J’ B > f2 or J A / J’ B >= f2), the NN filter model may not be applied.

[0484] viii. In one example, when the cost J A and the cost J’ B the ratio between is less than and / or equal to the threshold value (J A / J’ B < f3 or J A / J’ B <= f3), the NN filter model may not be applied.

[0485] ix. In one example, J A / J’ B > f2 || J A / J’ B < f3.

[0486] x. In one example, the above-mentioned inventions can be combined in any way.

[0487] xi. In the above-mentioned inventions, the parameters f0, f1, f2 or f3 can be set according to the time domain layer.

[0488] xii. In the above-mentioned inventions, the parameters f0, f1, f2 or f3 can be set according to the QP (quantization parameter).

[0489] xiii. In the above-mentioned inventions, the parameters f0, f1, f2 or f3 can be set according to the slice type.

[0490] xiv. In the above-mentioned inventions, the parameters f0, f1, f2 or f3 can be set according to the configuration (e.g., all intra-frame, random access, low-delay B, low-delay P, etc.).

[0491] e. In one example, during the RDO process, NN filtering can be applied only to partial samples of the block.

[0492] i. For example, in a W×H block, NN filtering can be applied only to the central W1×H1 sub-block.

[0493] 1) For example, W1 = W / 2, H1 = H / 2.

[0494] 2) For example, W1 = 3*W / 4, H1 = 3*H / 4.

[0495] f. Whether and / or how to utilize the NN filter model during the RDO process can depend on the time domain layer.

[0496] i. In one example, the NN filter model can be applied to the time domain layers with ID greater than K.

[0497] 1) In one example, K is equal to 0, 1, 2, 3, 4, 5, 6.

[0498] ii. In one example, the NN filter model can be applied to the time domain layer with ID less than K.

[0499] 1) In one example, K is equal to 0, 1, 2, 3, 4, 5, 6.

[0500] g. Whether and / or how to utilize the NN filter model in the RDO process may depend on the codec statistics of the sub-codec unit.

[0501] i. In one example, when the NN filter model is applied to part or all of the sub-coding units, the NN filter model may not be applied or may be applied to the current block.

[0502] ii. In one example, when rate-distortion cost and / or distortion of some or all sub-codec units are available, the NN filter model may not be applied or may be applied to the current block.

[0503] 5. Examples

[0504] 5.1. Example #1

[0505] In this implementation, the in-loop filtering based on convolutional neural network with adaptive model selection (DAM) is extended to the rate-distortion optimization (RDO) process. And the number of residual blocks in DAM is reduced to 4. DAM is applied to the codec unit level to select the best partition structure based on the RDO criterion. The rate-distortion cost can be formulated as:

[0506] J=D+lambda*R

[0507] Where D represents the minimum value of distortion with and without DAM.

[0508] Before applying DAM, check the cost of partitioning pattern A J A and the cost J of partitioning mode B B . DAM is skipped when the following conditions are met.

[0509] J A >f0*J B ||J B >f1*J A

[0510] Among them, f0 and f1 are parameters.

[0511] Embodiments of the present disclosure relate to an RDO process with a machine learning model. As used herein, the term "machine learning model" may also be referred to as a "machine learning model". The machine learning model may include any kind of model, such as a neural network (NN) model (also referred to as an "NN filter" or "NN filter model"), a convolutional neural network (CNN) model, and the like. Alternatively, in some embodiments, the machine learning model further includes a non-NN-based model or a non-NN-based filter. The scope of the present disclosure is not limited in this regard.

[0512] As used herein, the term "video unit" or "video block" may be a sequence, a picture, a slice, a tile, a brick, a sub-picture, a codec tree unit (CTU) / codec tree block (CTB), a CTU / CTB row, one or more codec units (CU) / codec blocks (CB), one or more CTU / CTBs, one or more virtual pipeline data units (VPDU), a sub-region within a picture / slice / tile / brick. As used herein, the term "parent video unit" may refer to a unit that is larger than a video unit. A parent unit will contain several video units. For example, if a video unit is a CTU, the parent unit may be a slice, a CTU row, multiple CTUs, etc.

[0513] Fig.16 A flow chart of a method 1600 for video processing according to some embodiments of the present disclosure is shown. The method 1600 may be implemented for conversion between a current video block of a video and a bitstream of the video. As used herein, the term "current video block" may also be referred to as a "target video block".

[0514] like Fig.16 As shown, at block 1610, a distortion value of a current video block is determined based on a set of distortion metrics. The set of distortion metrics includes at least one item: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model.

[0515] Method 1600 enables determining a distortion value of a current video block based on a set of distortion metrics. The determined distortion value can be applied in a rate-distortion optimization (RDO) process, and thus the RDO process can be improved. In this way, codec effectiveness and codec efficiency can be improved.

[0516] At block 1620, a conversion is performed based on the distortion metric. In some embodiments, the conversion may include encoding the current video block into a bitstream. Alternatively or additionally, the conversion may include decoding the current video block from a bitstream.

[0517] In some embodiments, at block 1610, if it is determined that the first machine learning model is applied to at least one of the plurality of candidate modes for the current video block, the first distortion metric is determined as the distortion value. In some embodiments, at block 1610, if it is determined that the first machine learning model is not applied to at least one of the plurality of candidate modes for the current video block, the third distortion metric is determined as the distortion value.

[0518] In some embodiments, multiple candidate modes are not partition modes. That is, if all or one of the T candidate modes is not a partition mode, the distortion value in the RDO criterion can be a combination of the distortion with a model such as an NN filter model and the distortion without the NN filter model.

[0519] In some embodiments, when all or one of the T candidate modes is not a partition mode, D in the RDO criterion is a combination of the distortion with the pth machine learning model such as the NN filter model and / or the distortion without the NN filter model. If the pth NN filter model is not applied to one or all of the T candidate modes, D in the RDO criterion is the distortion without the NN filter model (D=D ORG ). If the p-th NN filter model is not applied to one or all of the T candidate modes, D in the RDO criterion is the distortion with the p-th NN filter model (D = D p NNLF ).

[0520] In some embodiments, at block 1610, if it is determined that the second machine learning model is applied to at least one of the multiple candidate modes of the current video block, a minimum distortion metric of the second distortion metric and the third distortion metric is determined as the distortion value. In some embodiments, at block 1610, if it is determined that the second machine learning model is not applied to at least one of the multiple candidate modes of the current video block, a distortion value is determined based on the first distortion metric and the third distortion metric.

[0521] In some embodiments, if it is determined that the first machine learning model is applied to multiple candidate modes, the first distortion metric is determined as the distortion value. In some embodiments, if it is determined that the first machine learning model is not applied to multiple candidate modes, the third distortion metric is determined as the distortion value.

[0522] In some embodiments, at block 1610, a distortion value may be determined based on a combination of a set of distortion metrics. The combination of the set of distortion metrics may be determined based on at least one of: codec statistics of the current video block, a first use of a first machine learning model, a second use of a second machine learning model, or a priority order of a set of distortion metrics.

[0523] In some embodiments, the method 1600 further comprises: determining the priority order based on at least one of: a codec mode of the current video block, or codec statistics of the current video block.

[0524] In some embodiments, the codec statistics include at least one of the following: a prediction mode of the current video block, a type of prediction mode, a quantization parameter (QP) of the current video block, a temporal layer of the current video block, or a slice type of the current video block.

[0525] In some embodiments, the fourth distortion metric includes the minimum distortion metric of the second distortion metric and the third distortion metric, and the priority of the fourth distortion metric is higher than the priority of the first distortion metric. That is, the minimum distortion value D=min(D q NNLF , D ORG ) can have a distortion D=D greater than that with the p-th NN filter model p NNLF Higher priority.

[0526] In some embodiments, the first distortion metric has a higher priority than the third distortion metric. That is, the distortion D with the pth NN filter model = D p NNLF It can have a distortion D=D better than that without the NN filter model ORG Higher priority.

[0527] In some embodiments, the plurality of candidate modes for the current video block are partition modes, and the combination of a set of distortion metrics includes a first distortion metric, a second distortion metric, and a third distortion metric.

[0528] In some embodiments, the first machine learning model includes one of: a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF).

[0529] In some embodiments, the second machine learning model comprises a convolutional neural network (CNN) model.

[0530] In some embodiments, the first machine learning model is the same as the second machine learning model.

[0531] In some embodiments, the first index of the first machine learning model is the same as the second index of the second machine learning model.

[0532] In some embodiments, the method 1600 further includes: performing a rate-distortion optimization (RDO) process on the current video block based on the distortion value.

[0533] In some embodiments, the RDO process may be performed on multiple candidate modes based on rate-distortion costs. The rate-distortion cost may be determined based on the sum of the distortion value and a weighted rate of the current video block.

[0534] In some embodiments, the number of the plurality of candidate modes includes one of: 1, 2, 3, or 4.

[0535] In some embodiments, the rate-distortion cost can be determined based on the distortion metric by using the following formula: J=D+lambda*R (referred to as the RDO criterion), where J represents the rate-distortion cost, D represents the distortion value, R represents the rate of the current video block, and lambda represents a predefined factor.

[0536] In some embodiments, the target codec tool of the current video block can be determined based on the rate-distortion cost. As used herein, the term "target codec tool" can refer to the codec tool used by the current video block during conversion. The codec tool can be a codec mode or a codec method.

[0537] In some embodiments, a set of distortion metrics also includes at least one of: a third distortion metric and a minimum distortion metric among a fifth distortion metric determined based on one of a plurality of machine learning models, the plurality of machine learning models including a first machine learning model and a second machine learning model, a sixth distortion metric determined based on a predefined model or a selected model among the plurality of machine learning models, a scaled metric of the third distortion metric, or a weighted sum of the first distortion metric, the second distortion metric, the third distortion metric, the fifth distortion metric, and the sixth distortion metric.

[0538] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device, according to which a distortion value of a current video block of the video is determined based on a set of distortion metrics. The set of distortion metrics includes at least one of the following: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model. The bitstream is generated based on the distortion value.

[0539] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. According to the method, a distortion value of a current video block of the video is determined based on a set of distortion metrics. The set of distortion metrics includes at least one of the following: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model. A bitstream is generated based on the distortion value. The bitstream is stored in a non-transitory computer-readable recording medium.

[0540] According to an embodiment of the present disclosure, a distortion value may be determined based on a set of distortion metrics. The determined distortion value may be applied in an RDO process. In this way, the RDO process may be improved, and thus the codec effectiveness and codec efficiency may be improved.

[0541] Fig.17 1 is a flow chart of a method 1700 for video processing according to some embodiments of the present disclosure. The method 1700 may be implemented for conversion between a current video block of a video and a bit stream of the video. Fig.17 As shown in , at block 1710, it is determined whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for a current video block. At block 1720, the RDO process is performed on the current video block based on the determination.

[0542] At block 1730, conversion is performed based on the RDO process. In some embodiments, conversion may include encoding the current video block into a bitstream. Alternatively or additionally, conversion may include decoding the current video block from a bitstream.

[0543] By using method 1700, it can be determined whether to apply a machine learning model in the RDO process of the codec tool for the current video block, thereby improving the codec effectiveness and codec efficiency.

[0544] As used herein, the term "codec tool" or "target codec tool" may refer to a codec tool used by a current video block during conversion. A codec tool may be a codec mode or a codec method.

[0545] In some embodiments, a priority order of the codec tool and the other codec tool may be determined at block 1710. Based on the priority order, it may be determined whether to apply the machine learning model in the RDO process for the codec tool or for the other codec tool.

[0546] In some embodiments, if the priority order indicates that the first priority of the codec tool is higher than or equal to the second priority of the other codec tool, it is determined to apply the machine learning model in the RDO process for the codec tool. Alternatively or additionally, in some embodiments, if the priority order indicates that the first priority of the codec tool is lower than the second priority of the other codec tool, it is determined to apply the machine learning model in the RDO process for the other codec tool.

[0547] In some embodiments, if the first rate-distortion cost of the codec tool is greater than or equal to the second rate-distortion cost of the other codec tool, the first priority of the codec tool is determined to be higher than the second priority of the other codec tool. The first rate-distortion cost and the second rate-distortion cost are determined without using a machine learning model.

[0548] In some embodiments, if a first rate-distortion cost of a codec tool is less than or equal to a second rate-distortion cost of another codec tool, it is determined that the first priority of the codec tool is higher than the second priority of the other codec tool, and the first rate-distortion cost and the second rate-distortion cost are determined without using a machine learning model.

[0549] In some embodiments, the first rate-distortion cost or the second rate-distortion cost is determined based on a sum of a distortion value of the current video block and a weighted rate of the current video block.

[0550] In some embodiments, the distortion value for the current video block comprises a first distortion value determined without using a machine learning model.

[0551] In some embodiments, the priority order may be determined based on at least one of a codec mode of the current video block or codec statistics of the current video block.

[0552] In some embodiments, the codec statistics include at least one of the following: a prediction mode of the current video block, a type of prediction mode, a quantization parameter (QP) of the current video block, a temporal layer of the current video block, or a slice type of the current video block.

[0553] In some embodiments, at block 1710, a determination may be made whether to apply a machine learning model in an RDO process for a codec based on a first rate-distortion cost of the codec and a second rate-distortion cost of another codec. The first rate-distortion cost is determined without using the machine learning model. The second rate-distortion cost is determined based on the machine learning model.

[0554] In some embodiments, the first rate-distortion cost is determined based on a sum of the first distortion value of the current video block and a weighted rate for the current video block.

[0555] In some embodiments, the first distortion value for the current video block is determined without using a machine learning model.

[0556] In some embodiments, the second rate-distortion cost is determined based on a sum of the second distortion value of the current video block and a weighted rate for the current video block.

[0557] In some embodiments, the second distortion value of the current video block includes one of: a minimum distortion value of a first distortion value determined without using a machine learning model and a third distortion value determined according to a machine learning model, or the first distortion value.

[0558] In some embodiments, the first rate-distortion cost or the second rate-distortion cost can be determined based on the distortion metric by using the following formula: J=D+lambda*R, where J represents the rate-distortion cost, D represents the distortion value, R represents the rate of the current video block, and lambda represents a predefined factor.

[0559] In some embodiments, the machine learning model includes at least one of: a neural network (NN) model, a convolutional neural network (CNN) model, or a non-NN based model.

[0560] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a video processing device. According to the method, it is determined whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for a current video block of the video. Based on the determination, the RDO process is performed on the current video block. And a bitstream is generated based on the RDO process.

[0561] According to some other embodiments of the present disclosure, a method for storing a bitstream of a video is provided. According to the method, it is determined whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for a current video block of the video. The RDO process is performed on the current video block based on the determination. A bitstream is generated based on the RDO process. The bitstream is stored in a non-transitory computer-readable recording medium.

[0562] According to an embodiment of the present disclosure, it is possible to determine whether to apply a machine learning model in the RDO process for a codec tool, thereby improving the RDO process and thus improving codec effectiveness and codec efficiency.

[0563] It should be understood that the above method 1600 and / or method 1700 can be used in combination or alone. Any suitable combination of these methods can be used. The scope of the present disclosure is not limited in this regard.

[0564] By using the methods 1600 and / or 1700 alone or in combination, the RDO process can be improved, thereby improving the coding effectiveness and coding efficiency.

[0565] Implementations of the present disclosure may be described in light of the appended clauses, and features thereof may be combined in any reasonable manner.

[0566] Clause 1. A method for video processing, comprising: for conversion between a current video block of a video and a bitstream of the video, determining a distortion value of the current video block based on a set of distortion metrics, the set of distortion metrics including at least one item: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; and performing the conversion based on the distortion value.

[0567] Clause 2. A method according to Clause 1, wherein determining the distortion value comprises: if it is determined that the first machine learning model is applied to at least one of the multiple candidate modes of the current video block, determining the first distortion metric as the distortion value; and if it is determined that the first machine learning model is not applied to at least one of the multiple candidate modes of the current video block, determining the third distortion metric as the distortion value.

[0568] Clause 3. The method of clause 2, wherein the plurality of candidate patterns are not partition patterns.

[0569] Clause 4. A method according to any one of clauses 1 to 3, wherein determining the distortion value comprises: if it is determined that the second machine learning model is applied to at least one of the multiple candidate modes of the current video block, determining the minimum distortion metric between the second distortion metric and the third distortion metric as the distortion value; and if it is determined that the second machine learning model is not applied to at least one of the multiple candidate modes of the current video block, determining the distortion value based on the first distortion metric and the third distortion metric.

[0570] Clause 5. A method according to Clause 4, wherein determining the distortion value based on the first distortion metric and the third distortion metric includes: if it is determined that the first machine learning model is applied to the multiple candidate modes, determining the first distortion metric as the distortion value; and if it is determined that the first machine learning model is not applied to the multiple candidate modes, determining the third distortion metric as the distortion value.

[0571] Clause 6. A method according to any one of clauses 1 to 5, wherein determining the distortion value comprises: determining the distortion value based on a combination of the set of distortion metrics, the combination of the set of distortion metrics being determined based on at least one of: encoding and decoding statistics of the current video block, a first use of the first machine learning model, a second use of the second machine learning model, or a priority order of the set of distortion metrics.

[0572] Clause 7. The method according to Clause 6 further includes: determining the priority order based on at least one of the following: a coding mode of the current video block, or coding statistics of the current video block.

[0573] Clause 8. A method according to clause 6 or 7, wherein the codec statistics include at least one of the following: a prediction mode of the current video block, a type of the prediction mode, a quantization parameter (QP) of the current video block, a temporal layer of the current video block, or a slice type of the current video block.

[0574] Clause 9. The method of any one of clauses 6 to 8, wherein a fourth distortion metric comprises a minimum distortion metric of the second distortion metric and the third distortion metric, the fourth distortion metric having a higher priority than the first distortion metric.

[0575] Clause 10. The method of any one of clauses 6 to 9, wherein the first distortion metric has a higher priority than the third distortion metric.

[0576] Clause 11. A method according to any one of clauses 6 to 10, wherein the multiple candidate modes for the current video block are partition modes; and the combination of the set of distortion metrics includes the first distortion metric, the second distortion metric, and the third distortion metric.

[0577] Clause 12. A method according to any one of clauses 1 to 11, wherein the first machine learning model comprises one of the following: a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF).

[0578] Clause 13. A method according to any one of clauses 1 to 12, wherein the second machine learning model comprises a convolutional neural network (CNN) model.

[0579] Clause 14. A method according to any one of clauses 1 to 13, wherein the first machine learning model is the same as the second machine learning model.

[0580] Clause 15. A method according to any one of clauses 1 to 14, wherein the first index of the first machine learning model is the same as the second index of the second machine learning model.

[0581] Clause 16. The method of any one of clauses 1 to 15, further comprising: performing a rate-distortion optimization (RDO) process on the current video block based on the distortion value.

[0582] Clause 17. The method of clause 16, wherein performing the RDO process on the current video block comprises performing the RDO process on a plurality of candidate modes based on a rate-distortion cost, wherein the rate-distortion cost is determined based on a sum of the distortion value and a weighted rate of the current video block.

[0583] Clause 18. The method of clause 17, wherein the number of the plurality of candidate patterns comprises one of: 1, 2, 3, or 4.

[0584] Clause 19. A method according to any one of clauses 1 to 18, wherein the set of distortion metrics also includes at least one of the following: the minimum distortion metric among the third distortion metric and a fifth distortion metric determined based on a machine learning model among a plurality of machine learning models, the plurality of machine learning models including the first machine learning model and the second machine learning model, a sixth distortion metric determined based on a predefined model or a selected model among the plurality of machine learning models, a scaled metric of the third distortion metric, or a weighted sum of the first distortion metric, the second distortion metric, the third distortion metric, the fifth distortion metric and the sixth distortion metric.

[0585] Clause 20. A method for video processing, comprising: determining, for conversion between a current video block of a video and a bitstream of the video, whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for the current video block; performing the RDO process on the current video block based on the determination; and performing the conversion based on the RDO process.

[0586] Clause 21. A method according to Clause 20, wherein determining whether to apply a machine learning model in the RDO process for the codec tool comprises: determining a priority order of the codec tool and another codec tool; and determining whether to apply the machine learning model in the RDO process for the codec tool or for the other codec tool based on the priority order.

[0587] Clause 22. A method according to Clause 21, wherein determining whether to apply the machine learning model in the RDO process for the codec tool or for the other codec tool based on the priority order comprises: if it is determined that the priority order indicates that the first priority of the codec tool is higher than or equal to the second priority of the other codec tool, determining to apply the machine learning model in the RDO process for the codec tool; and if it is determined that the priority order indicates that the first priority of the codec tool is lower than the second priority of the other codec tool, determining to apply the machine learning model in the RDO process for the other codec tool.

[0588] Clause 23. A method according to clause 21 or 22, wherein determining the priority order includes: if it is determined that the first rate-distortion cost of the codec tool is greater than or equal to the second rate-distortion cost of the other codec tool, determining that the first priority of the codec tool is higher than the second priority of the other codec tool, and the first rate-distortion cost and the second rate-distortion cost are determined without using the machine learning model.

[0589] Clause 24. A method according to clause 21 or 22, wherein determining the priority order includes: if it is determined that the first rate-distortion cost of the codec tool is less than or equal to the second rate-distortion cost of the other codec tool, determining that the first priority of the codec tool is higher than the second priority of the other codec tool, and the first rate-distortion cost and the second rate-distortion cost are determined without using the machine learning model.

[0590] Clause 25. The method of clause 23 or 24, wherein the first rate-distortion cost or the second rate-distortion cost is determined based on a sum of a distortion value of the current video block and a weighted rate of the current video block.

[0591] Clause 26. The method of clause 25, wherein the distortion value of the current video block comprises a first distortion value determined without using the machine learning model.

[0592] Clause 27. The method of any one of clauses 21 to 26, wherein determining the priority order comprises determining the priority order based on at least one of a codec mode of the current video block or codec statistics of the current video block.

[0593] Clause 28. A method according to clause 27, wherein the codec statistics include at least one of the following: a prediction mode of the current video block, a type of the prediction mode, a quantization parameter (QP) of the current video block, a temporal layer of the current video block, or a slice type of the current video block.

[0594] Clause 29. A method according to any one of clauses 20 to 28, wherein determining whether to apply the machine learning model in the RDO process for the codec tool comprises: determining whether to apply the machine learning model in the RDO process for the codec tool based on a first rate-distortion cost of the codec tool and a second rate-distortion cost of another codec tool, the first rate-distortion cost being determined without using the machine learning model, and the second rate-distortion cost being determined according to the machine learning model.

[0595] Clause 30. The method of Clause 29, wherein the first rate-distortion cost is determined based on a sum of a first distortion value of the current video block and a weighted rate for the current video block.

[0596] Clause 31. The method of clause 30, wherein the first distortion value of the current video block is determined without using the machine learning model.

[0597] Clause 32. The method of any of clauses 29 to 31, wherein the second rate-distortion cost is determined based on a sum of a second distortion value for the current video block and a weighted rate for the current video block.

[0598] Clause 33. A method according to Clause 32, wherein the second distortion value of the current video block comprises one of: the minimum distortion value of a first distortion value determined without using the machine learning model and a third distortion value determined according to the machine learning model, or the first distortion value.

[0599] Clause 34. A method according to any one of clauses 20 to 33, wherein the machine learning model comprises at least one of the following: a neural network (NN) model, a convolutional neural network (CNN) model, or a non-NN based model.

[0600] Clause 35. The method of any one of clauses 1 to 34, wherein the converting comprises encoding the current video block into the bitstream.

[0601] Clause 36. The method of any one of clauses 1 to 34, wherein the converting comprises decoding the current video block from the bitstream.

[0602] Clause 37. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of clauses 1 to 36.

[0603] Clause 38. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any one of clauses 1 to 36.

[0604] Clause 39. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: determining a distortion value of a current video block of the video based on a set of distortion metrics, the set of distortion metrics comprising at least one of the following: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; and generating the bitstream based on the distortion value.

[0605] Clause 40. A method for storing a bitstream of a video, comprising: determining a distortion value of a current video block of the video based on a set of distortion metrics, the set of distortion metrics including at least one of: a first distortion metric determined according to a first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; and generating the bitstream based on the distortion value; and storing the bitstream in a non-transitory computer-readable recording medium.

[0606] Clause 41. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for a current video block of the video; performing the RDO process on the current video block based on the determination; and generating the bitstream based on the RDO process.

[0607] Clause 42. A method for storing a bitstream of a video, comprising: determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for a current video block of the video; performing the RDO process on the current video block based on the determination; generating the bitstream based on the RDO process; and storing the bitstream in a non-transitory computer-readable recording medium.

[0608] Example Device

[0609] Fig.18 A block diagram of a computing device 1800 in which various embodiments of the present disclosure may be implemented is shown. The computing device 1800 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0610] It should be understood that Fig.18 The computing device 1800 shown in the figure is for illustration purposes only and is not intended to in any way imply any limitation on the functionality and scope of the embodiments of the present disclosure.

[0611] like Fig.18 As shown, computing device 1800 includes a general computing device 1800. Computing device 1800 may include at least one or more processors or processing units 1810, memory 1820, storage unit 1830, one or more communication units 1840, one or more input devices 1850, and one or more output devices 1860.

[0612] In some embodiments, computing device 1800 can be implemented as any user terminal or server terminal with computing power. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet computer, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, and includes the accessories and peripherals of these devices, or any combination thereof. It is conceivable that computing device 1800 can support any type of interface to the user (such as a "wearable" circuit device, etc.).

[0613] The processing unit 1810 may be a physical processor or a virtual processor and may implement various processes based on a program stored in the memory 1820. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 1800. The processing unit 1810 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0614] The computing device 1800 typically includes various computer storage media. Such media can be any media accessible by the computing device 1800, including but not limited to volatile media and non-volatile media, or removable media and non-removable media. The memory 1820 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory) or any combination thereof. The storage unit 1830 can be any removable or non-removable medium, and can include machine-readable media, such as a memory, a flash drive, a disk, or other media that can be used to store information and / or data and can be accessed in the computing device 1800.

[0615] The computing device 1800 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Fig.18 Although not shown in the figure, a magnetic disk drive for reading from and / or writing to a removable nonvolatile magnetic disk and an optical disk drive for reading from and / or writing to a removable nonvolatile optical disk may be provided. In this case, each drive may be connected to the bus (not shown) via one or more data medium interfaces.

[0616] The communication unit 1840 communicates with another computing device via a communication medium. In addition, the functions of the components in the computing device 1800 can be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 1800 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general network nodes.

[0617] The input device 1850 may be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. The output device 1860 may be one or more of various output devices, such as a display, a speaker, a printer, etc. With the help of the communication unit 1840, the computing device 1800 may also communicate with one or more external devices (not shown), such as storage devices and display devices, and the computing device 1800 may also communicate with one or more devices that enable a user to interact with the computing device 1800, or any device that enables the computing device 1800 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), if necessary. Such communication may be performed via an input / output (I / O) interface (not shown).

[0618] In some embodiments, some or all components of computing device 1800 may also be arranged in a cloud computing architecture rather than being integrated in a single device. In a cloud computing architecture, components may be provided remotely and work together to implement the functions described in the present disclosure. In some embodiments, cloud computing provides computing, software, data access and storage services, which will not require the end user to know the physical location or configuration of the system or hardware that provides these services. In various embodiments, cloud computing provides services via a wide area network (e.g., the Internet) using a suitable protocol. For example, a cloud computing provider provides applications via a wide area network, which can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture and the corresponding data may be stored on a remote server. The computing resources in a cloud computing environment may be merged or distributed at the location of a remote data center. Cloud computing infrastructure may provide services through a shared data center, although they appear as a single access point for users. Therefore, the cloud computing architecture may be used to provide the components and functions described herein from a service provider at a remote location. Alternatively, they may be provided by a conventional server, or installed directly or otherwise on a client device.

[0619] In an embodiment of the present disclosure, the computing device 1800 may be used to implement video encoding / decoding. The memory 1820 may include one or more video encoding / decoding modules 1825 having one or more program instructions. These modules can be accessed and executed by the processing unit 1810 to perform the functions of the various embodiments described herein.

[0620] In an example embodiment of performing video encoding, the input device 1850 may receive video data as input 1870 to be encoded. The video data may be processed by, for example, the video codec module 1825 to generate an encoded bitstream. The encoded bitstream may be provided as output 1880 via the output device 1860.

[0621] In an example embodiment of performing video decoding, the input device 1850 may receive an encoded bitstream as input 1870. The encoded bitstream may be processed by, for example, the video codec module 1825 to generate decoded video data. The decoded video data may be provided as output 1880 via the output device 1860.

[0622] Although the present disclosure has been specifically shown and described with reference to the preferred embodiments of the present disclosure, it will be understood by those skilled in the art that various changes can be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. These changes are intended to be encompassed by the scope of the present application. Therefore, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, a distortion value of the current video block is determined based on a set of distortion metrics, the set of distortion metrics comprising at least one item: a first distortion metric determined according to the first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; as well as The converting is performed based on the distortion value.

2. The method of claim 1 , wherein determining the distortion value comprises: If it is determined that the first machine learning model is applied to at least one of the plurality of candidate modes of the current video block, determining the first distortion metric as the distortion value; as well as If it is determined that the first machine learning model is not applied to at least one of the multiple candidate modes of the current video block, the third distortion metric is determined as the distortion value. The method according to claim 2 , wherein the plurality of candidate patterns are not partition patterns.

4. The method according to any one of claims 1 to 3, wherein determining the distortion value comprises: If it is determined that the second machine learning model is applied to at least one of the plurality of candidate modes of the current video block, determining a minimum distortion metric of the second distortion metric and the third distortion metric as the distortion value; as well as If it is determined that the second machine learning model is not applied to at least one of the multiple candidate modes of the current video block, the distortion value is determined based on the first distortion metric and the third distortion metric.

5. The method of claim 4, wherein determining the distortion value based on the first distortion metric and the third distortion metric comprises: If it is determined that the first machine learning model is applied to the plurality of candidate modes, determining the first distortion metric as the distortion value; as well as If it is determined that the first machine learning model is not applied to the multiple candidate modes, the third distortion metric is determined as the distortion value.

6. The method according to any one of claims 1 to 5, wherein determining the distortion value comprises: determining the distortion value based on a combination of the set of distortion metrics, The combination of the set of distortion metrics is determined based on at least one of: encoding and decoding statistics of the current video block, a first use of the first machine learning model, a second use of the second machine learning model, or A priority order of the set of distortion metrics.

7. The method according to claim 6, further comprising: The priority order is determined based on at least one of the following: The encoding and decoding mode of the current video block, or, The encoding and decoding statistics of the current video block.

8. The method according to claim 6 or 7, wherein the encoding and decoding statistics include at least one of the following: the prediction mode of the current video block, the type of prediction mode, the quantization parameter (QP) of the current video block, the temporal layer of the current video block, or The slice type of the current video block.

9. The method according to any one of claims 6 to 8, wherein a fourth distortion metric comprises a minimum distortion metric of the second distortion metric and the third distortion metric, and a priority of the fourth distortion metric is higher than a priority of the first distortion metric.

10. The method according to any one of claims 6 to 9, wherein the first distortion metric has a higher priority than the third distortion metric.

11. The method according to any one of claims 6 to 10, wherein the plurality of candidate modes for the current video block are partition modes; and The combination of the set of distortion metrics includes the first distortion metric, the second distortion metric, and the third distortion metric.

12. The method of any one of claims 1 to 11, wherein the first machine learning model comprises one of the following: Deblocking filter, Sample Adaptive Offset (SAO), or Adaptive loop filter (ALF).

13. A method according to any one of claims 1 to 12, wherein the second machine learning model comprises a convolutional neural network (CNN) model.

14. A method according to any one of claims 1 to 13, wherein the first machine learning model is the same as the second machine learning model.

15. A method according to any one of claims 1 to 14, wherein the first index of the first machine learning model is the same as the second index of the second machine learning model.

16. The method according to any one of claims 1 to 15, further comprising: A rate-distortion optimization (RDO) process is performed on the current video block based on the distortion value.

17. The method of claim 16, wherein performing the RDO process on the current video block comprises: The RDO process is performed on a plurality of candidate modes based on a rate-distortion cost, wherein the rate-distortion cost is determined based on a sum of the distortion value and a weighted rate of the current video block. The method according to claim 17 , wherein the number of the plurality of candidate patterns comprises one of the following: 1, 2, 3 or 4.

19. The method according to any one of claims 1 to 18, wherein the set of distortion metrics further comprises at least one of the following: a minimum distortion metric among the third distortion metric and a fifth distortion metric determined according to one of a plurality of machine learning models, the plurality of machine learning models including the first machine learning model and the second machine learning model, a sixth distortion metric determined according to a predefined model or a selected model among the plurality of machine learning models, a scaled measure of the third distortion metric, or A weighted sum of the first distortion metric, the second distortion metric, the third distortion metric, the fifth distortion metric, and the sixth distortion metric.

20. A method for video processing, comprising: For conversion between a current video block of a video and a bitstream of the video, determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec tool for the current video block; performing the RDO process on the current video block based on the determination; as well as The converting is performed based on the RDO process.

21. The method of claim 20, wherein determining whether to apply a machine learning model in the RDO process for the codec tool comprises: Determining a priority order between the codec tool and another codec tool; as well as A determination is made based on the priority order whether to apply the machine learning model in the RDO process for the codec tool or for the other codec tool.

22. The method of claim 21, wherein determining whether to apply the machine learning model in the RDO process for the codec tool or for the other codec tool based on the priority order comprises: If it is determined that the priority order indicates that the first priority of the codec tool is higher than or equal to the second priority of the other codec tool, determine to apply the machine learning model in the RDO process for the codec tool; as well as If it is determined that the priority order indicates that the first priority of the codec tool is lower than the second priority of the other codec tool, it is determined to apply the machine learning model in the RDO process for the other codec tool.

23. The method of claim 21 or 22, wherein determining the priority order comprises: If it is determined that the first rate-distortion cost of the codec tool is greater than or equal to the second rate-distortion cost of the other codec tool, it is determined that the first priority of the codec tool is higher than the second priority of the other codec tool, and the first rate-distortion cost and the second rate-distortion cost are determined without using the machine learning model.

24. The method of claim 21 or 22, wherein determining the priority order comprises: If it is determined that the first rate-distortion cost of the codec tool is less than or equal to the second rate-distortion cost of the other codec tool, it is determined that the first priority of the codec tool is higher than the second priority of the other codec tool, and the first rate-distortion cost and the second rate-distortion cost are determined without using the machine learning model.

25. The method of claim 23 or 24, wherein the first rate-distortion cost or the second rate-distortion cost is determined based on a sum of a distortion value of the current video block and a weighted rate of the current video block.

26. The method of claim 25, wherein the distortion value for the current video block comprises a first distortion value determined without using the machine learning model.

27. The method according to any one of claims 21 to 26, wherein determining the priority order comprises: The priority order is determined based on at least one of a codec mode of the current video block or codec statistics of the current video block.

28. The method of claim 27, wherein the codec statistics include at least one of the following: the prediction mode of the current video block, the type of prediction mode, the quantization parameter (QP) of the current video block, the temporal layer of the current video block, or The slice type of the current video block.

29. The method of any one of claims 20 to 28, wherein determining whether to apply the machine learning model in the RDO process for the codec tool comprises: Based on a first rate-distortion cost of the codec tool and a second rate-distortion cost of another codec tool, determine whether to apply the machine learning model in the RDO process for the codec tool, the first rate-distortion cost being determined without using the machine learning model, and the second rate-distortion cost being determined according to the machine learning model.

30. The method of claim 29, wherein the first rate-distortion cost is determined based on a sum of a first distortion value for the current video block and a weighted rate for the current video block.

31. The method of claim 30, wherein the first distortion value for the current video block is determined without using the machine learning model.

32. The method of any one of claims 29 to 31, wherein the second rate-distortion cost is determined based on a sum of a second distortion value of the current video block and a weighted rate of the current video block.

33. The method of claim 32, wherein the second distortion value of the current video block comprises one of: a minimum distortion value of a first distortion value determined without using the machine learning model and a third distortion value determined according to the machine learning model, or The first distortion value.

34. The method of any one of claims 20 to 33, wherein the machine learning model comprises at least one of: Neural Network (NN) Model, Convolutional Neural Network (CNN) models, or Non-NN based models.

35. The method of any one of claims 1 to 34, wherein the converting comprises encoding the current video block into the bitstream.

36. The method of any one of claims 1 to 34, wherein the converting comprises decoding the current video block from the bitstream.

37. An apparatus for processing video data, comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 36.

38. A non-transitory computer-readable storage medium storing instructions, the instructions causing a processor to execute the method according to any one of claims 1 to 36.

39. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: Determining a distortion value for a current video block of the video based on a set of distortion metrics, the set of distortion metrics comprising at least one of: a first distortion metric determined according to the first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; as well as The bitstream is generated based on the distortion values.

40. A method for storing a bitstream of a video, comprising: Determining a distortion value for a current video block of the video based on a set of distortion metrics, the set of distortion metrics comprising at least one of: a first distortion metric determined according to the first machine learning model, a second distortion metric determined according to a second machine learning model, or a third distortion metric determined without using the first machine learning model and the second machine learning model; as well as generating the bitstream based on the distortion value; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

41. A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, wherein the method comprises: determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec for a current video block of the video; performing the RDO process on the current video block based on the determination; as well as The bitstream is generated based on the RDO process.

42. A method for storing a bitstream of a video, comprising: determining whether to apply a machine learning model in a rate-distortion optimization (RDO) process of a codec for a current video block of the video; performing the RDO process on the current video block based on the determination; generating the bitstream based on the RDO process; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images