Method and device for video processing and medium

Abstract

The embodiment of the invention provides a video processing scheme. A method for video processing is presented. The method comprises: applying a connection to a plurality of filters for a transition between a video unit of a video and a bitstream of the video; applying the plurality of filter combinations with the connection to the video unit; and performing conversion based on the filtered video unit.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to video processing techniques, and more specifically to a combination of neural network (NN) based filters for image / video encoding and decoding. Background Art

[0002] Nowadays, digital video capabilities are being applied to all aspects of people's lives. Various types of video compression technologies have been proposed for video encoding / decoding, such as MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264 / MPEG-4 Part 10 Advanced Video Codec (AVC), ITU-TH.265 High Efficiency Video Codec (HEVC) standard, and Versatile Video Codec (VVC) standard. However, it is generally expected to further improve the encoding and decoding efficiency of video encoding and decoding technologies. Public Content

[0003] Embodiments of the present disclosure provide a solution for video processing.

[0004] In a first aspect, a method for video processing is provided. The method comprises: for conversion between a video unit of a video and a bitstream of the video, applying a connection to a plurality of filters; applying a combination of the plurality of filters with the connection to the video unit; and performing conversion based on the filtered video unit. The method according to the first aspect of the present disclosure adaptively combines filters, which can advantageously improve coding efficiency and performance.

[0005] In a second aspect, a device for video processing is provided. The device comprises a processor and a non-volatile memory having instructions thereon. The instructions, when executed by the processor, cause the processor to perform the method according to the first aspect of the present disclosure.

[0006] In a third aspect, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions, wherein the instructions enable a processor to execute the method according to the first aspect of the present disclosure.

[0007] In a fourth aspect, another non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by an apparatus for video processing. The method includes: applying a connection to a plurality of filters; applying a plurality of filter combinations with the connection to a video unit of the video; and generating a bitstream based on the filtered video unit.

[0008] In a fifth aspect, a method for storing a bitstream of a video is provided, the method comprising: applying a connection to a plurality of filters; applying the plurality of filter combinations with the connection to a video unit of the video; generating a bitstream based on the filtered video unit; and storing the bitstream in a non-transitory computer-readable recording medium.

[0009] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become more apparent through the following detailed description with reference to the accompanying drawings.In the exemplary embodiments of the present disclosure, the same reference numerals generally refer to the same components.

[0011] Figure 1 A block diagram showing an example video encoding and decoding system according to some embodiments of the present disclosure is shown;

[0012] Figure 2 A block diagram of a first example video encoder according to some embodiments of the present disclosure is shown;

[0013] Figure 3 A block diagram of an example video decoder according to some embodiments of the present disclosure is shown;

[0014] Figure 4 An example diagram showing an example of raster scan striping of a picture;

[0015] Figure 5 An example diagram showing an example of rectangular striping of a picture;

[0016] Figure 6 An example diagram showing examples of a picture divided into tiles, bricks, and rectangular strips;

[0017] Fig. 7A An example diagram showing a CTB that spans a bottom picture boundary;

[0018] Figure 7B An example diagram showing a CTB that crosses the right picture boundary;

[0019] Figure 7C An example diagram showing a CTB that spans the lower right picture boundary;

[0020] Figure 8 An example diagram showing an example of an encoder block diagram;

[0021] Fig. 9An example diagram showing picture samples on an 8×8 grid and horizontal and vertical block boundaries and non-overlapping blocks of 8×8 samples;

[0022] Fig.10 An example diagram showing pixels involved in filter on / off decisions and strong / weak filter selection;

[0023] FIG. 11A to FIG. 11D An example diagram of four one-dimensional orientation maps used for EO sample classification is shown;

[0024] FIG. 12A to FIG. 12C An example diagram showing an example of a GALF filter shape;

[0025] FIG. 13A to FIG. 13C An example diagram showing an example of a relative coordinator for 5x5 diamond filter support;

[0026] Fig.14 An example diagram showing an example of relative coordinates for a 5×5 diamond filter support;

[0027] Fig.15A An example diagram showing the architecture of the proposed CNN filter is shown, and Fig. 15B An example diagram showing the construction of a ResBlock (residual block) in a CNN filter;

[0028] FIG. 16A to FIG. 16E Schematic diagrams of filters with connections according to embodiments of the present disclosure are respectively shown;

[0029] Fig.17A and Fig. 17B Schematic diagrams of samples according to embodiments of the present disclosure are respectively shown;

[0030] Fig.18 A flowchart showing a method for video processing according to an embodiment of the present disclosure; and

[0031] Fig.19 A block diagram of a computing device is shown in which various embodiments of the present disclosure may be implemented.

[0032] Throughout the drawings, the same or similar reference numbers generally refer to the same or similar elements. DETAILED DESCRIPTION

[0033] The principle of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are only for illustrative purposes, and help those skilled in the art to understand and implement the present disclosure, without implying any limitation to the scope of the present disclosure. In addition to the methods described below, the disclosure described herein can also be implemented in various ways.

[0034] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0035] References in this disclosure to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment includes the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an example embodiment, whether or not explicitly described, it is considered to be within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in relation to other embodiments.

[0036] It should be understood that although the terms "first" and "second" etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the example embodiments, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0037] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "including", "having", and / or "comprising" when used herein indicate the presence of the features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. Example Environment

[0038] Figure 1 1 is a block diagram illustrating an example video codec system 100 that may utilize the techniques of the present disclosure. As shown, the video codec system 100 may include a source device 110 and a destination device 120. The source device 110 may also be referred to as a video encoding device, and the destination device 120 may also be referred to as a video decoding device. In operation, the source device 110 may be configured not to generate encoded video data, and the destination device 120 may be configured not to decode the encoded video data generated by the source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input / output (I / O) interface 116.

[0039] Video source 112 may include a source such as a video capture device. Examples of a video capture device include, but are not limited to, an interface that receives video data from a video content provider, a computer graphics system for generating video data, and / or combinations thereof.

[0040] The video data may include one or more pictures. The video encoder 114 encodes the video data from the video source 112 to generate a bit stream. The bit stream may include a bit sequence that forms a coded representation of the video data. The bit stream may include a coded picture and associated data. The coded picture is a coded representation of the picture. The associated data may include a sequence parameter set, a picture parameter set, and other grammatical structures. The I / O interface 116 may include a modulator / demodulator and / or a transmitter. The coded video data may be directly transmitted to the destination device 120 via the I / O interface 116 through the network 130A. The coded video data may also be stored on a storage medium / server 130B for access by the destination device 120.

[0041] The destination device 120 may include an I / O interface 126, a video decoder 124, and a display device 122. The I / O interface 126 may include a receiver and / or a modem. The I / O interface 126 may obtain encoded video data from the source device 110 or the storage medium / server 130B. The video decoder 124 may decode the encoded video data. The display device 122 may display the decoded video data to the user. The display device 122 may be integrated with the destination device 120, or may be outside the destination device 120, which is configured not to be connected to an external display device interface.

[0042] The video encoder 114 and the video decoder 124 may operate according to a video compression standard, such as the High Efficiency Video Codec (HEVC) standard, the Versatile Video Codec (VVC) standard, and other existing and / or further standards.

[0043] Figure 2 is a block diagram showing an example of a video encoder 200 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video encoder 114 in the system 100 is shown.

[0044] Video encoder 200 may be configured to implement any or all of the techniques of this disclosure. Figure 2 In the example of , video encoder 200 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video encoder 200. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0045] In some embodiments, the video encoder 200 may include a partitioning unit 201, a prediction unit 202, a residual generation unit 207, a transformation unit 208, a quantization unit 209, an inverse quantization unit 210, an inverse transformation unit 211, a reconstruction unit 212, a buffer 213 and an entropy coding unit 214, and the prediction unit 202 may include a mode selection unit 203, a motion estimation unit 204, a motion compensation unit 205 and an intra-frame prediction unit 206.

[0046] In other examples, the video encoder 200 may include more, fewer, or different functional components. In an example, the prediction unit 202 may include an intra-block copy (IBC) unit. The IBC unit may perform prediction in an IBC mode, where at least one reference picture is a picture where the current video block is located.

[0047] Furthermore, although some components (eg, motion estimation unit 204 and motion compensation unit 205) may be integrated, for purposes of explanation, these components are described in detail below. Figure 2 are shown separately in the example.

[0048] The partitioning unit 201 may partition a picture into one or more video blocks. The video encoder 200 and the video decoder 300 may support various video block sizes.

[0049] The mode selection unit 203 may select one of a plurality of coding modes (intra or inter), for example, based on the error result, and provide the resulting intra-coded block or inter-coded block to the residual generation unit 207 to generate residual block data, and to the reconstruction unit 212 to reconstruct the coded block for use as a reference picture. In some examples, the mode selection unit 203 may select a combination of intra and inter prediction (CIIP) modes, where the prediction is based on an inter prediction signal and an intra prediction signal. In the case of inter prediction, the mode selection unit 203 may also select a resolution for the motion vector (e.g., sub-pixel precision or integer pixel precision) for the block.

[0050] To perform inter-frame prediction on the current video block, the motion estimation unit 204 may generate motion information for the current video block by comparing the current video block with one or more reference frames from the buffer 213. The motion compensation unit 205 may determine a predicted video block for the current video block based on the motion information and decoded samples of pictures from the buffer 213 other than the picture associated with the current video block.

[0051] The motion estimation unit 204 and the motion compensation unit 205 may perform different operations on the current video block, for example, depending on whether the current video block is in an I slice, a P slice, or a B slice. As used herein, an "I slice" may refer to a portion of a picture consisting of macroblocks, all of which are based on macroblocks within the same picture. Furthermore, as used herein, in some aspects, a "P slice" and a "B slice" may refer to a portion of a picture consisting of macroblocks that are independent of macroblocks in the same picture.

[0052] In some examples, the motion estimation unit 204 may perform unidirectional prediction on the current video block, and the motion estimation unit 204 may search the reference pictures of list 0 or list 1 to find the reference video block for the current video block. The motion estimation unit 204 may then generate a reference index and a motion vector, the reference index indicating the reference picture in list 0 or list 1 containing the reference video block, and the motion vector indicating the spatial displacement between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as the motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the reference video block indicated by the motion information of the current video block.

[0053] Alternatively, in other examples, the motion estimation unit 204 may perform bidirectional prediction on the current video block. The motion estimation unit 204 may search the reference pictures in list 0 for a reference video block for the current video block, and may also search the reference pictures in list 1 for another reference video block for the current video block. The motion estimation unit 204 may then generate a plurality of reference indexes and a plurality of motion vectors, the plurality of reference indexes indicating a plurality of reference pictures in list 0 and list 1 containing a plurality of reference video blocks, and the plurality of motion vectors indicating a plurality of spatial displacements between the plurality of reference video blocks and the current video block. The motion estimation unit 204 may output the plurality of reference indexes and the plurality of motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block for the current video block based on the plurality of reference video blocks indicated by the motion information of the current video block.

[0054] In some examples, motion estimation unit 204 may output a complete set of motion information for use in a decoding process by a decoder. Alternatively, in some embodiments, motion estimation unit 204 may signal motion information of a current video block with reference to motion information of another video block. For example, motion estimation unit 204 may determine that motion information of a current video block is sufficiently similar to motion information of a neighboring video block.

[0055] In one example, motion estimation unit 204 may indicate to video decoder 300 a value in a syntax structure associated with the current video block that indicates that the current video block has the same motion information as another video block.

[0056] In another example, the motion estimation unit 204 may identify another video block and a motion vector difference (MVD) in a syntax structure associated with the current video block. The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the indicated video block. The video decoder 300 may use the motion vector of the indicated video block and the motion vector difference to determine the motion vector of the current video block.

[0057] As discussed above, the video encoder 200 may signal motion vectors in a predictive manner.Two examples of prediction signaling techniques that may be implemented by the video encoder 200 include Advanced Motion Vector Prediction (AMVP) and merge mode signaling.

[0058] The intra prediction unit 206 may perform intra prediction on the current video block. When the intra prediction unit 206 performs intra prediction on the current video block, the intra prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same picture. The prediction data for the current video block may include a prediction video block and various syntax elements.

[0059] The residual generation unit 207 may generate residual data for the current video block by subtracting (e.g., indicated by a minus sign) the prediction video block(s) of the current video block from the current video block. The residual data of the current video block may include residual video blocks corresponding to different sample portions of samples in the current video block.

[0060] In other examples, such as in skip mode, there may be no residual data for the current video block, and the residual generation unit 207 may not perform a subtraction operation.

[0061] Transform processing unit 208 may generate one or more transform coefficient video blocks for a current video block by applying one or more transforms to the residual video block associated with the current video block.

[0062] After transform processing unit 208 generates a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values ​​associated with the current video block.

[0063] The inverse quantization unit 210 and the inverse transform unit 211 may apply inverse quantization and inverse transform to the transform coefficient video block, respectively, to reconstruct a residual video block from the transform coefficient video block. The reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more prediction video blocks generated by the prediction unit 202 to generate a reconstructed video block associated with the current video block for storage in the buffer 213.

[0064] After reconstruction unit 212 reconstructs the video block, a loop filtering operation may be performed to reduce video blocking artifacts in the video block.

[0065] The entropy coding unit 214 may receive data from other functional components of the video encoder 200. When the entropy coding unit 214 receives data, the entropy coding unit 214 may perform one or more entropy coding operations to generate entropy coded data and output a code stream including the entropy coded data.

[0066] Figure 3 is a block diagram showing an example of a video decoder 300 according to some embodiments of the present disclosure, which may be Figure 1 An example of a video decoder 124 in the system 100 is shown.

[0067] Video decoder 300 may be configured to perform any or all of the techniques of this disclosure. Figure 3 In the example of , video decoder 300 includes multiple functional components. The techniques described in this disclosure can be shared between the various components of video decoder 300. In some examples, a processor can be configured to perform any or all of the techniques described in this disclosure.

[0068] exist Figure 3 In the example of , the video decoder 300 includes an entropy decoding unit 301, a motion compensation unit 302, an intra prediction unit 303, an inverse quantization unit 304, an inverse transform unit 305, and a reconstruction unit 306 and a buffer 307. In some examples, the video decoder 300 can perform a decoding process that is generally opposite to the encoding process described with respect to the video encoder 200.

[0069] The entropy decoding unit 301 can retrieve the encoded code stream. The encoded code stream may include entropy encoded video data (e.g., encoded video data blocks). The entropy decoding unit 301 can decode the entropy encoded video data, and the motion compensation unit 302 can determine motion information from the entropy decoded video data, the motion information including motion vectors, motion vector precision, reference picture list indexes, and other motion information. The motion compensation unit 302 can determine the information, for example, by performing AMVP and merge mode. AMVP is used, including deriving several most likely candidates based on data and reference pictures of neighboring PBs. The motion information typically includes horizontal and vertical motion vector displacement values, one or two reference picture indexes, and in the case of prediction areas in B stripes, also includes an identification of which reference picture list is associated with each index. As used herein, in some aspects, "merge mode" may refer to deriving motion information from blocks that are adjacent in space or time.

[0070] The motion compensation unit 302 may generate motion compensated blocks, possibly performing interpolation based on interpolation filters.Identifiers for the interpolation filters used with sub-pixel precision may be included in the syntax elements.

[0071] The motion compensation unit 302 may calculate interpolated values ​​for sub-integer pixels of a reference block using interpolation filters used by the video encoder 200 during encoding of the video block. The motion compensation unit 302 may determine the interpolation filters used by the video encoder 200 based on received syntax information, and the motion compensation unit 302 may use the interpolation filters to generate a prediction block.

[0072] The motion compensation unit 302 may use at least part of the syntax information to determine the size of blocks used to encode (multiple) frames and / or (multiple) slices of the encoded video sequence, partition information describing how each macroblock of a picture of the encoded video sequence is partitioned, a mode indicating how each partition is encoded, one or more reference frames (and reference frame lists) for each inter-coded block, and other information for decoding the encoded video sequence. As used herein, in some aspects, a "slice" may refer to a data structure that can be decoded independently of other slices of the same picture in terms of entropy coding, signal prediction, and residual signal reconstruction. A slice may be an entire picture, or it may be a region of a picture.

[0073] The intra prediction unit 303 may use, for example, an intra prediction mode received in a bitstream to form a prediction block from spatially neighboring blocks. The inverse quantization unit 304 inversely quantizes (i.e., dequantizes) the quantized video block coefficients provided in the bitstream and decoded by the entropy decoding unit 301. The inverse transform unit 305 applies an inverse transform.

[0074] The reconstruction unit 306 may obtain the decoded block, for example, by adding the residual block to the corresponding prediction block generated by the motion compensation unit 302 or the intra prediction unit 303. If necessary, a deblocking filter may also be applied to filter the decoded block in order to remove blocking artifacts. The decoded video block is then stored in a buffer 307, which provides reference blocks for subsequent motion compensation / intra prediction, and the buffer 307 also generates the decoded video for presentation on a display device.

[0075] Some exemplary embodiments of the present disclosure will be described in detail below. It should be noted that the section titles used in this document are for ease of understanding, rather than limiting the embodiments disclosed in the section to that section. In addition, although some embodiments are described with reference to general video codecs or other specific video codecs, the disclosed technology is also applicable to other video codec technologies. In addition, although some embodiments describe the video encoding steps in detail, it should be understood that the corresponding decoding steps of the unencoding will be implemented by the decoder. In addition, the term video processing includes video encoding or compression, video decoding or decompression, and video transcoding, in which video pixels are represented from one compression format to another compression format or at different compression bit rates. 1. Brief Overview The present disclosure relates to video coding technology. Specifically, it relates to a loop filter in image / video coding. It can be applied to existing video coding standards, such as High Efficiency Video Codec (HEVC), Universal Video Codec (VVC) or a standard to be completed (e.g., AVS3). It can also be applied to future video coding standards or video codecs, or used as a post-processing method outside the encoding / decoding process. 2. Introduction Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO / IEC standards. The ITU-T produced H.261 and H.263, the ISO / IEC produced MPEG-1 and MPEG-4 Vision, and the two organizations jointly produced the H.262 / MPEG-2 Video and H.264 / MPEG-4 Advanced Video Codec (AVC) and H.265 / HEVC standards. Since H262, video coding standards are based on a hybrid video codec structure that uses temporal prediction plus transform codecs. In order to explore future video codec technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, JVET has taken many new methods and put them into reference software called the Joint Exploration Model (JEM). In April 2018, a Joint Video Experts Group (JVET) between VCEG (Q6 / 16) and ISO / IEC JTC1SC29 / WG11 (MPEG) was created to work on the VVC standard targeting a 50% bitrate reduction compared to HEVC. VVC version 1 was completed in July 2020. The latest version of the VVC draft, Versatile Video Codec (Draft 10), can be found at: http: / / phenix.it-sudparis.eu / jvet / doc_end_user / current_document.php?id=10399. The latest reference software for VVC (called VTM) can be found at: https: / / vcgit.hhi.fraunhofer.de / jvet / VVCSoftware_VTM / - / tags / VTM-10.0. 2.1 Color Space and Chroma Subsampling A color space, also called a color model (or color system), is an abstract mathematical model that simply describes the range of colors as a tuple of numbers, usually 3 or 4 values ​​or color components (such as RGB). Basically, a color space is a refinement of a coordinate system and subspace. For video compression, the most frequently used color spaces are YCbCr and RGB. YCbCr, Y'CbCr or Y Pb / Cb Pr / Cr (also written as YCBCR or Y'CBCR) is a family of color spaces used as part of the color imaging pipeline in video and digital photography systems. Y' is the luma component, CB and CR are the blue difference and red difference chroma components. Y' (with superscript notation) is different from Y, which is the brightness, meaning that the light intensity is encoded non-linearly based on the gamma-corrected RGB primaries. Chroma subsampling is the practice of encoding an image by achieving less resolution for chroma information than for luminance information, taking advantage of the human visual system's lower acuity for color differences than for luminance. 2.1.1 4:4:4 Each of the three Y'CbCr components has the same sampling rate, so there is no chroma subsampling. This scheme is sometimes used in high-end film scanners and movie post-production. 2.1.2 4:2:2 The two chroma components are sampled at half the sampling rate of luma: the horizontal chroma resolution is halved. This reduces the bandwidth of the uncompressed video signal by one third with little visual difference. 2.1.3 4:2:0 In 4:2:0, the horizontal sampling is doubled compared to 4:1:1, but the vertical resolution is halved because the Cb and Cr channels are sampled only on every alternate line in this scheme. Therefore, the data rate is the same. Cb and Cr are each subsampled horizontally and vertically at a factor of 2. There are three variants of the 4:2:0 scheme with different horizontal and vertical settings. In MPEG-2, Cb and Cr are co-located horizontally. Cb and Cr are located between pixels in the vertical direction (placed with gaps). In JPEG / JFIF, H261, and MPEG-1, Cb and Cr are placed with gaps in between alternating luma samples. In 4:2:0 DV, Cb and Cr are co-located in the horizontal direction. In the vertical direction, they are co-located on alternate lines. 2.2 Definition of Video Unit A picture is divided into one or more tile rows and one or more tile columns. A tile is a sequence of CTUs covering a rectangular area of ​​a picture. A tile is divided into one or more bricks, each brick consisting of multiple CTU rows within the tile. A tile that is not divided into multiple bricks is also called a brick. However, a brick that is a true subset of a tile is not called a tile. A strip contains multiple tiles of an image or multiple bricks of a tile. Two modes of strips are supported, raster scan strip mode and rectangular strip mode. In raster scan strip mode, a strip contains a sequence of tiles in a raster scan of tiles of a picture. In rectangular strip mode, a strip contains multiple bricks of a picture that together form a rectangular region of the picture. The bricks within a rectangular strip are in the order of the raster scan of the bricks of the strip. Figure 4 An example diagram 400 shows an example of raster scan striping of a picture. Figure 4In , the image is divided into 12 tiles and 3 raster scan strips. Figure 4 A picture with 18×12 luma CTUs in is divided into 12 tiles and 3 raster scan strips (with information). Figure 5 An example diagram 500 shows an example of a rectangular striping division of a picture. Figure 5 In , the image is divided into 24 tiles (6 tile columns and 4 tile rows) and 9 rectangular strips. Figure 5 A picture with 18×12 luma CTUs is divided into 24 tiles and 9 rectangular strips (with information). Figure 6 An example diagram 600 shows an example of a picture divided into tiles, bricks, and rectangular strips. Figure 6 In , the image is divided into 4 tiles (2 tile columns and 2 tile rows), 11 bricks (the upper left tile contains 1 brick, the upper right tile contains 5 bricks, the lower left tile contains 2 bricks, and the lower right tile contains 3 bricks), and 4 rectangular strips. Figure 6 The image in is divided into 4 tiles, 11 bricks and 4 rectangular strips (with information). 2.2.1 CTU / CTB Size In VVC, the CTU size signaled in the SPS by the syntax element log2_ctu_size_minus2 can be as small as 4x4. 7.3.2.3 Sequence parameter set RBSP syntax log2_ctu_size_minus2 plus 2 specifies the luma codec tree block size for each CTU. log2_min_luma_coding_block_size_minus2 plus 2 specifies the minimum luma coding block size. Derive the following variables: CtbLog2SizeY, CtbSizeY, MinCbLog2SizeY, MinCbSizeY, MinTbLog2SizeY, MaxTbLog2SizeY, MinTbSizeY, MaxTbSizeY, PicWidthInCtbsY, PicHeightInCtbsY, PicSizeInCtbsY, PicWidthInMinCbsY, PicHeightInMinCbsY, PicSizeInMinCbsY, PicSizeInSamplesY, PicWidthInSamplesC, and PicHeightInSamplesC: CtbLog2SizeY = log2_ctu_size_minus2 + 2 (7 - 9) CtbSizeY = 1 << CtbLog2SizeY (7 - 10) MinCbLog2SizeY = log2_min_luma_coding_block_size_minus2 + 2 (7 - 11) MinCbSizeY = 1 << MinCbLog2SizeY (7 - 12) MinTbLog2SizeY = 2 (7 - 13) MaxTbLog2SizeY = 6 (7 - 14) MinTbSizeY = 1 << MinTbLog2SizeY (7 - 15) MaxTbSizeY = 1 << MaxTbLog2SizeY (7 - 16) PicWidthInCtbsY = Ceil(pic_width_in_luma_samples ÷ CtbSizeY) (7 - 17) PicHeightInCtbsY = Ceil(pic_height_in_luma_samples ÷ CtbSizeY) (7 - 18) PicSizeInCtbsY = PicWidthInCtbsY * PicHeightInCtbsY (7 - 19) PicWidthInMinCbsY = pic_width_in_luma_samples / MinCbSizeY (7 - 20) PicHeightInMinCbsY = pic_height_in_luma_samples / MinCbSizeY (7 - 21) PicSizeInMinCbsY = PicWidthInMinCbsY * PicHeightInMinCbsY (7 - 22) PicSizeInSamplesY = pic_width_in_luma_samples * pic_height_in_luma_samples (7 - 23) PicWidthInSamplesC = pic_width_in_luma_samples / SubWidthC (7 - 24) PicHeightInSamplesC = pic_height_in_luma_samples / SubHeightC (7 - 25) 2.2.2 CTUs in the Picture Assume the CTB / LCU size indicated by M x N (usually M equals N, as defined in HEVC / VVC), and for the CTBs located at the boundaries of the picture (or tile or stripe or other types of regions, taking the picture boundary as an example), K x L samples are within the picture boundary, where K < M or L < N. Fig. 7A An example figure 700 showing a CTB spanning the bottom picture boundary is presented, where K = M and L < N. Figure 7B An example figure 720 showing a CTB spanning the right picture boundary is presented, where K < M and L = N. Figure 7C An example figure 740 showing a CTB spanning the bottom - right picture boundary is presented, where K < M and L < N. For Figures 7A-7C those CTBs depicted in 2.3 Encoding and Decoding Streams of Typical Video Codecs Figure 8An example diagram 800 of an example of an encoder block diagram for VVC is shown, which contains three in-loop filtering blocks: a deblocking filter (DF) 805, a sample adaptive offset (SAO) 806, and an ALF 807. Unlike the DF 805, which uses a predefined filter, the SAO 806 and the ALF 807 utilize the original samples of the current picture to reduce the mean square error between the original samples and the reconstructed samples by adding an offset and by applying a finite impulse response (FIR) filter, respectively, where the offset and the filter coefficients are signaled by the codec side information. The ALF 807 is located at the last processing stage for each picture and can be seen as a tool that attempts to capture and repair artifacts created by previous stages. 2.4 Deblocking Filter (DB) The input to DB is the reconstructed samples before the loop filter. First, filter the vertical edges in the picture. Then, use the samples modified by the vertical edge filtering process as input to filter the horizontal edges in the picture. The vertical and horizontal edges in the CTB of each CTU are processed separately on a codec unit basis. The vertical edges of the codec blocks in the codec unit are filtered starting from the edge on the left side of the codec block in its geometric order toward the right side of the codec block. The horizontal edges of the codec blocks in the codec unit are filtered starting from the edge on the top of the codec block in its geometric order toward the bottom of the codec block. Fig. 9 An example diagram 900 shows picture samples on an 8x8 grid along with horizontal and vertical block boundaries and non-overlapping blocks of 8x8 samples that may be blocked in parallel. 2.4.1 Boundary Decision The filter is applied on 8x8 block boundaries. In addition, it must be a transform block boundary or a codec subblock boundary (e.g. due to the use of affine motion prediction (ATMVP)). For those codecs that are not such boundaries, the filter is disabled. 2.4.2 Boundary strength calculation For a transform block boundary / codec subblock boundary, if it lies in the 8x8 grid, it can be filtered and the bS[xD i ][yD j ](where [xD i ][yD j ] represents coordinates) are defined in Table 1 and Table 2 respectively. Table 1 Boundary Strength (When SPS IBC is disabled) Table 2 Boundary Strength (When SPS IBC is Enabled) 2.4.3 Deblocking decision for luminance component The deblocking decision process is described in this subsection. Fig.10 An example diagram 1000 of pixels involved in filter on / off decisions and strong / weak filter selection is shown. The wider-stronger luminance filter is used only if conditions 1, 2, and 3 are all true. Condition 1 is the "large block condition". This condition detects whether the samples on the P side and Q side belong to large blocks, represented by the variables bSidePisLargeBlk and bSideQisLargeBlk, respectively. bSidePisLargeBlk and bSideQisLargeBlk are defined as follows. bSidePisLargeBlk=((edge ​​type is vertical and p 0 Belongs to CUs with width >= 32)||(edge ​​type is horizontal and p 0 Belongs to CU with height >= 32))? True: False bSideQisLargeBlk=((edge ​​type is vertical and q 0 Belongs to CUs with width >=32)||(edge ​​type is horizontal and q 0 Belongs to CU with height >=32))? True: False Based on bSidePisLargeBlk and bSideQisLargeBlk, condition 1 is defined as follows. Condition 1 = (bSidePisLargeBlk || bSidePisLargeBlk)? True: False Next, if condition 1 is true, condition 2 will be checked further. First, the following variables are exported: – dp0, dp3, dq0, dq3 are first derived as in HEVC. –if (side p is greater than or equal to 32) dp0=(dp0+Abs(p5 0 -2*p4 0 +p3 0 )+1)>>1 dp3=(dp3+Abs(p5 3 -2*p4 3 +p3 3 )+1)>>1 –if (q side is greater than or equal to 32) dq0=(dq0+Abs(q5 0 -2*q4 0 +q3 0)+1)>>1 dq3=(dq3+Abs(q5 3 -2*q4 3 +q3 3 )+1)>>1 Condition 2 = (d < β)? True: False Where d=dp0+dq0+dp3+dq3. If condition 1 and condition 2 are valid, then further check if any block uses sub-blocks: Finally, if both conditions 1 and 2 are valid, the proposed deblocking method will check condition 3 (large block strong filter condition), which is defined as follows. In condition 3StrongFilterCondition, export the following variables: dpq is derived as in HEVC. sp 3 =Abs(p 3 -p 0 ), as derived in HEVC. if (side p is greater than or equal to 32) if(Sp==5) sp 3 =(sp 3 +Abs(p 5 -p 3 )+1)>>1 else sp 3 =(sp 3 +Abs(p 7 -p 3 )+1)>>1 sq 3 =Abs(q 0 -q 3 ), is exported as in HEVC if (q side is greater than or equal to 32) If(Sq==5) sq 3 =(sq 3 +Abs(q 5 -q 3 )+1)>>1 else sq 3 =(sq 3 +Abs(q 7 -q 3 )+1)>>1. As in HEVC, StrongFilterCondition = (dpq is less than (β>>2), sp 3 +sq 3 Less than (3*β>>5), and Abs(p 0 -q 0 ) is less than (5*t C +1)>>1)? True: False. 2.4.4 Stronger Deblocking Filter for Luma (Designed for Larger Blocks) When samples on either side of the boundary belong to a large block, a bilinear filter is used. Samples belonging to a large block are defined as when the width of a vertical edge is >= 32, and when the height of a horizontal edge is >= 32. The bilinear filters are listed below. Block boundary samples p for i=0 to Sp-1 i and block boundary samples p for j = 0 to Sq-1 i (pi and qi are the i-th samples in a row for filtering vertical edges, or in a column for filtering horizontal edges) are then replaced by linear interpolation as follows: —p i ′=(fi*Middle s,t +(64-f i )*P s +32)>>6), crop to p i ±tcPD i —q j ′=(g j *Middle s,t +(64-g j )*Q s +32)v>6), clipped to q j ±tcPD j Among them, the term tcPD i and tcPD j is the position-dependent clipping described in Section 2.4.7, and is given below g j ,f i ,Middle s,t ,P s and Q s : 2.4.5 Deblocking Control for Chroma A chroma strong filter is used on both sides of the block boundary. Here, the chroma filter is selected when both sides of the chroma edge are greater than or equal to 8 (chroma position), and the decision meets the following three conditions: The first condition is a decision for boundary strength and large blocks. The proposed filter can be applied when the block width or height orthogonally across the block edge is equal to or greater than 8 in the chroma sample domain. The second and third conditions are basically the same for HEVC luminance deblocking decisions, which are an on / off decision and a strong filter decision, respectively. In the first decision, the boundary strength (bS) is modified for chroma filtering, and the conditions are checked sequentially. If a condition is met, the remaining conditions with lower priority are skipped. When bS is equal to 2, chroma deblocking is performed, or bS is equal to 1 when a large block boundary is detected. The second and third conditions are essentially the same as the HEVC luma strong filter decision as follows. In the second condition: d is then derived as in HEVC luma deblocking. When d is less than β, the second condition will be true. In the third condition, StrongFilterCondition is derived as follows: dpq is derived as in HEVC. sp 3 =Abs(p 3 -p 0 ), as derived in HEVC. sq 3 =Abs(q 0 -q 3 ), as derived in HEVC. As in the HEVC design, StrongFilterCondition = (dpq is less than (β>>2), sp 3 +sq 3 Less than (β>>3), and Abs(p 0 -q 0 ) is less than (5*t C +1)>>1). 2.4.6 Strong Deblocking Filter for Chroma The following strong deblocking filter for chroma is defined as: p 2 ′=(3*p 3 +2*p 2 +p 1 +p 0 +q 0 +4)>>3 p 1′=(2*p 3 +p 2 +2*p 1 +p 0 +q 0 +q 1 +4)>>3 p 0 ′=(p 3 +p 2 +p 1 +2*p 0 +q 0 +q 1 +q 2 +4)>>3 The proposed chroma filter performs deblocking on a 4x4 chroma sample grid. 2.4.7 Position-dependent cropping Position dependent clipping tcPD is applied to the output samples of the luma filtering process involving strong and long filters that modify 7, 5 and 3 samples at the boundaries. Assuming the quantization error distribution, it is suggested to increase the clipping value for samples that are expected to have higher quantization noise and therefore higher deviation of the reconstructed sample values ​​from the true sample value. For each P or Q boundary filtered with the asymmetric filter, depending on the outcome of the decision making process in Section 2.4.2, a position-dependent threshold table is selected from two tables (i.e., Tc7 and Tc3 listed below) that are provided to the decoder as side information: Tc7={6,5,4,3,2,1,1}; Tc3={6,4,2}; tcPD=(Sp==3)? Tc3:Tc7; tcQD=(Sq==3)? Tc3:Tc7; For P or Q boundaries filtered with a short symmetric filter, a lower magnitude position-dependent threshold is applied: Tc3 = {3, 2, 1}; After defining the threshold, the filtered p' is clipped according to the tcP and tcQ values. i and q' i Sample values ​​are clipped: p" i = Clip3(p' i +tcP i ,p' i –tcP i ,p' i ); q" j = Clip3(q' j +tcQ j,q' j –tcQ j ,q' j ); where p' i and q' i is the filtered sample value, p" i and q" j is the output sample value after clipping, and tcP i tcP i is the clipping threshold derived from the VVC tc parameters and tcPD and tcQD. Function Clip3 is the clipping function as specified in VVC. 2.4.8 Sub-block Deblocking Adjustment To achieve parallel-friendly deblocking using long filters and sub-block deblocking, the long filter is restricted to modify at most 5 samples on one side using sub-block deblocking (AFFINE or ATMVP or DMVR), as shown in the luma control for the long filter. In addition, sub-block deblocking is adjusted so that sub-block boundaries on the 8x8 grid close to the CU or implicit TU boundary are restricted to modify at most two samples on each side. The following applies to sub-block boundaries that are not aligned with CU boundaries. In case edge equal to 0 corresponds to CU boundary, edge equal to 2 or equal to orthogonalLength-2 corresponds to sub-block boundary 8 samples from CU boundary etc. where implicit-TU is true if implicit splitting of TU is used. 2.5SAO The input of SAO is the reconstructed samples after DB. The concept of SAO is to reduce the average sample distortion of a region by first classifying the region samples into multiple categories with a selected classifier, obtaining an offset for each category, and then adding the offset to each sample of the category, where the classifier index and the offset of the region are encoded and decoded in the bitstream. In HEVC and VVC, the region (the unit for SAO parameter signaling) is defined as CTU. Two SAO types are used in HEVC to meet the low complexity requirements. These two types are edge offset (EO) and band offset (BO), which will be discussed in further detail below. The index of the SAO type is encoded and decoded (it is in the range of [0,2]). For EO, ​​sample classification is based on the comparison between the current sample and the adjacent samples according to a one-dimensional directional map: horizontal, vertical, 135° diagonal, and 45° diagonal. Fig.11A An example graph 1100 of a one-dimensional directional map of EO sample classification with level (EO1 category=0) is shown. Fig. 11BAn example graph 1120 of a one-dimensional directional map with vertical (EO category = 1) classification of EO samples is shown. Fig. 11C An example graph 1140 of a one-dimensional directional map of EO sample classification with a 135° diagonal (EO class = 2) is shown. Fig.11D An example graph 1160 of a one-dimensional directional map of EO sample classification with a 45° diagonal (EO class = 3) is shown. For a given EO classification, each sample within the CTB is classified into one of five categories. The current sample value, labeled "c", is compared to its two neighbors along the selected 1D map. The classification rules for each sample are summarized in Table 1. Categories 1 and 4 are associated with local valleys and local peaks along the selected 1D map, respectively. Categories 2 and 3 are associated with concave corners and convex corners along the selected 1D map, respectively. If the current sample does not belong to EO category 1 to 4, then it is category 0 and SAO is not applied. Table 3: Sample classification rules for edge offset category condition 1 c < a and c < b 2 (c<a&&c==b)||(c==a&&c<b) 3 (c>a&&c==b)||(c==a&&c>b) 4 c>a&&c>b 5 None of the above 2.6 Adaptive Loop Filter Based on Geometric Transformation in JEM The input of DB is the reconstructed samples after DB and SAO. The sample classification and filtering process is based on the reconstructed samples after DB and SAO. In JEM, a geometric transform based adaptive loop filter (GALF) with block-based filter adaptation is applied. For the luminance component, one of 25 filters is selected for each 2×2 block based on the direction and activity of the local gradient. 2.6.1 Filter Shape Fig. 12A An example diagram 1200 is shown showing an example of a GALF filter shape having a 5×5 diamond shape. Fig. 12B An example diagram 1220 is shown of an example of a GALF filter shape having a 7×7 diamond shape. Fig. 12C An example diagram 1240 is shown of an example of a GALF filter shape having a 9×9 diamond shape. In JEM, up to three diamond filter shapes can be selected for the luminance component (e.g. Figures 12A-12C ). An index is signaled at the picture level to indicate the filter shape for the luma component. Each square represents a sample, and Ci (i is 0 to 6 (left), 0 to 12 (middle), 0 to 20 (right)) represents the coefficient to be applied to the sample. For the chroma components in a picture, a 5×5 diamond shape is always used. 2.6.1.1 Block Classification Each 2×2 block is classified into one of 25 classes. The class index C is based on its directionality D and the quantized value of activity And is exported as follows: To calculate D and First, use the 1-D Laplacian to calculate the gradient in the horizontal, vertical, and two diagonal directions: The indices i and j refer to the coordinates of the top left sample in the 2x2 block, and R(i,j) indicates the reconstructed sample at coordinates (i,j). Then the maximum and minimum values ​​of the gradient D in the horizontal and vertical directions are set to: And the maximum and minimum values ​​of the gradients in the two diagonal directions are set to: To derive the value of the directivity D, these values ​​are compared with each other and with two threshold values ​​t 1 and t 2 For comparison: Step 1: If and are both true, then D is set to true. Step 2: If Then continue from step 3; otherwise, continue from step 4. Step 3: If Then D is set to 2; otherwise D is set to 1. Step 4: If Then D is set to 4; otherwise, D is set to 3. The activity value A is calculated as: A is further quantized to a range of 0 to 4, inclusive, and the quantized values ​​are expressed as For the two chroma components in a picture, no classification method is applied, ie, a single set of ALF coefficients is applied for each chroma component. 2.6.1.2 Geometric transformation of filter coefficients Fig.13A An example diagram 1300 of relative coordinators for a 5x5 diamond filter support (diagonal) is shown. Fig. 13B An example diagram 1320 of relative coordinators for 5×5 diamond filter support (vertically flipped) is shown. Fig. 13C An example diagram 1340 of relative coordinators for a 5x5 diamond filter support (rotation) is shown. Before filtering each 2×2 block, geometric transformations such as rotation or diagonal and vertical flipping are applied to the filter coefficients associated with the coordinates (k, l) according to the gradient values ​​calculated for that block. This is equivalent to applying these transformations to the samples in the filter support region. The idea is to make different blocks with ALF applied more similar by aligning the directionality. Three geometric transformations are introduced, including diagonal, vertical flip and rotation: Diagonal: f D (k,l)=f(l,k), Flip vertically: f V (k,l)=f(k,Kl-1) (9) Rotation: f R (k,l)=f(Kl-1,k) Where K is the size of the filter and 0≤k, l≤K-1 are the coefficient coordinates, so that the position (0,0) is in the upper left corner and the position (K-1,K-1) is in the lower right corner. The transform is applied to the filter coefficients f(k,l) according to the gradient values ​​calculated for the block. The relationship between the transform and the four gradients in the four directions is summarized in Table 4. FIG. 12A to FIG. 12C The transform coefficients for each position based on a 5x5 diamond are shown. Table 4 Mapping of gradients calculated for a block and transformation Gradient Value Deformation <![CDATA[g d2 <g d1 And g h <g v ]]> No transformation <![CDATA[g d2 <g d1 and g v <g h ]]> diagonal <![CDATA[g d1 <g d2 And g h <g v ]]> Flip Vertically <![CDATA[g d1 <g d2 And g v <g h ]]> Rotation 2.6.1.3 Filter parameter signaling In JEM, GALF filter parameters are signaled for the first CTU (i.e., after the slice header and before the SAO parameters of the first CTU). Up to 25 sets of luminance filter coefficients can be signaled. In order to reduce bit overhead, filter coefficients of different classifications can be merged. In addition, the GALF coefficients of the reference picture are stored and allowed to be reused as the GALF coefficients of the current picture. The current picture can choose to use the GALF coefficients stored for the reference picture and bypass the GALF coefficient signaling. In this case, only the index of one of the reference pictures is signaled, and the stored GALF coefficients of the indicated reference picture are inherited for the current picture. To support GALF temporal prediction, a candidate list of GALF filter sets is maintained. At the beginning of decoding a new sequence, the candidate list is empty. After decoding a picture, the corresponding filter set can be added to the candidate list. Once the size of the candidate list reaches the maximum allowed value (i.e., 6 in the current JEM), the new filter set overwrites the oldest set in decoding order, i.e., the first-in-first-out (FIFO) rule is applied to update the candidate list. In order to avoid duplication, the set can be added to the list only when the corresponding picture does not use GALF temporal prediction. In order to support temporal scalability, there are multiple candidate lists of filter sets, and each candidate list is associated with a temporal layer. More specifically, each array assigned by the temporal layer index (TempIdx) can constitute a filter set of a previously decoded picture equal to a lower TempIdx. For example, the kth array is assigned to be associated with a TempIdx equal to k, and it only contains filter sets from pictures with TempIdx less than or equal to k. After encoding or decoding a picture, the filter set associated with the picture will be used to update those arrays associated with equal or higher Templdx. Temporal prediction of the GALF coefficients is used for inter-frame coding frames to minimize signaling overhead. For intra-frames, temporal prediction is not available and a set of 16 fixed filters is assigned to each class. To indicate the use of fixed filters, a flag for each class is signaled and, if necessary, the index of the selected fixed filter is signaled. Even when a fixed filter is selected for a given class, the coefficients f(k, l) of the adaptive filter can still be sent for that class, in which case the coefficients of the filter to be applied to the reconstructed image are the sum of the two sets of coefficients. The filtering process of the luma component can be controlled at the CU level. A flag is signaled to indicate whether GALF is applied to the luma component of the CU. For chroma components, whether GALF is applied is indicated only at the picture level. 2.6.1.4 Filtering process At the decoder side, when GALF is enabled for a block, each sample R(i,j) within the block is filtered, producing a sample value R′(i,j) as shown below, where L represents the filter length and f m,n denotes the filter coefficients, and f(k,l) denotes the decoded filter coefficients. Fig.14 An example diagram 1400 showing an example of relative coordinates for a 5x5 diamond filter support. Fig.14An example of relative coordinates for a 5x5 diamond filter support assuming the coordinates (i, j) of the current sample are (0, 0) is shown. Samples in different coordinates filled with the same color are multiplied by the same filter coefficients. 2.7 Geometric Transformation-Based Adaptive Loop Filter (GALF) in VVC 2.7.1GALF in VTM-4 In VTM4.0, the filtering process of the adaptive loop filter is performed as follows: O(x,y)=∑ (i,j) w(i,j).I(x+i,y+j), (11) Where sample I(x+i,y+j) is the input sample, O(x,y) is the filtered output sample (ie, the filter result), and w(i,j) represents the filter coefficient. In fact, in VTM4.0, integer operations are used to implement fixed-point precision calculations: where L represents the filter length, and where w(i,j) are the filter coefficients in fixed-point precision. The current design of the GALF in the VVC has the following major changes compared to the GALF in the JEM: 1) Remove adaptive filter shapes. Only 7x7 filter shapes are allowed for luma components and only 5x5 filter shapes are allowed for chroma components. 2) Remove ALF parameters from slice / picture level to CTU level. 3) The calculation of the classification index is performed in the 4x4 level instead of the 2x2 level. In addition, the sub-sampled Laplacian calculation method for ALF classification is utilized. More specifically, there is no need to calculate the horizontal / vertical / 45 diagonal / 135 degree gradient for each sample within a block. Instead, 1:2 sub-sampling is used. 2.8 Nonlinear ALF in Current VVC 2.8.1 Filtering and Reshaping Equation (11) can be reformulated in the following expression without the effect of codec efficiency: O(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j).(I(x+i,y+j)-I(x,y)), (13) where w(i,j) are the same filter coefficients as in equation (11) [except that w(0,0) is equal to 1 in equation (13) and 1-∑ (i,j)≠(0,0) w(i,j)]. Using the above filter formula (13), VVC introduces nonlinearity to make the ALF more effective by using a simple clipping function, thereby reducing the influence of neighboring sample values ​​when the neighboring sample values ​​(I(x+i,y+j)) are significantly different from the current sample value being filtered (I(x,y)). More specifically, the ALF filter is modified as follows: O′(x,y)=I(x,y)+∑ (i,j)≠(0,0) w(i,j).K(I(x+i,y+j)-I(x,y),k(i,j)), (14) Where K(d,b)=min(b,max(-b,d)) is the clipping function and k(i,j) is the clipping parameter, which depends on the (i,j) filter coefficients. The encoder performs an optimization to find the best k(i,j). In some implementations, a clipping parameter k(i,j) is specified for each ALF filter, with one clipping value signaled per filter coefficient. This means that up to 12 clipping values ​​can be signaled in the bitstream for each luma filter, and up to 6 clipping values ​​for chroma filters. To limit the signaling cost and encoder complexity, only 4 fixed values ​​are used, which are the same for inter and intra slices. Since the variance of local differences for luma is typically higher than for chroma, two different sets of luma and chroma filters are applied. A maximum sample value in each set (here 1024 for a bit depth of 10 bits) is also introduced so that clipping can be disabled when it is not necessary. The set of clipping values ​​is provided in Table 5. The 4 values ​​have been chosen by splitting roughly equally in the logarithmic domain the full range of sample values ​​for luma (encoded on 10 bits) and the range of 4 to 1024 for chroma. More precisely, the brightness table of the clipped values ​​has been obtained by the following formula: Similarly, the chromaticity table for the clipped value is obtained according to the following formula: Table 5 Authorization clipping values The selected clipping value is encoded and decoded in the "alf_data" syntax element by using the Golomb coding scheme corresponding to the index of the clipping value in the above Table 5. This coding scheme is the same as that used for the filter index. 2.9 Bilateral In-Loop Filter 2.9.1 Bilateral Image Filter Bilateral image filter is a nonlinear filter that smooths noise while preserving edge structure. Bilateral filtering is a technique that makes the filter weight decrease not only with the distance between samples but also with the increase of intensity difference. In this way, the over-smoothing of edges can be improved. The weight is defined as: in △x and Δy are the distances in the vertical and horizontal directions, and ΔI is the intensity difference between the samples. The edge-preserving denoising bilateral filter uses a low-pass Gaussian filter for both the domain filter and the range filter. The domain low-pass Gaussian filter gives higher weights to pixels that are spatially close to the center pixel. The range low-pass Gaussian filter gives higher weights to pixels that are similar to the center pixel. Combining the range filter and the domain filter, the bilateral filter at the edge pixel becomes an elongated Gaussian filter oriented along the edge and greatly reduced in the direction of the gradient. This is why the bilateral filter can smooth noise while preserving edge structure. 2.9.2 Bilateral Filters in Video Codecs The bilateral filter in video codec is proposed as a codec tool for VVC. The filter acts as a loop filter in parallel with the sample adaptive offset (SAO) filter. Both the bilateral filter and SAO act on the same input samples, each filter produces an offset, and these offsets are then added to the input samples to produce output samples, which enter the next stage after cropping. The spatial filter strength σ d Determined by the block size, smaller blocks are filtered more strongly, and the intensity filter strength σ r Determined by the quantization parameter, where stronger filtering is used for higher QPs. Only the four closest samples are used, so the filtered sample strength I F Calculated as Among them I C Indicates the intensity of the central sample, △I A =I A -I C Represents the intensity difference between the center sample and the sample above. ΔI B , ΔI L and ΔI R Represent the intensity differences between the center sample and the samples below, to the left, and to the right, respectively. 2.10 Convolutional Neural Network-Based Loop Filters for Video Codecs 2.10.1 Convolutional Neural Networks In deep learning, convolutional neural networks (CNN or ConvNet) are a class of deep neural networks that are most commonly used to analyze visual images. They have very successful applications in image and video recognition / processing, recommendation systems, image classification, medical image analysis, and natural language processing. CNNs are regularized versions of multilayer perceptrons. Multilayer perceptrons usually refer to fully connected networks, i.e., each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks makes them prone to overfitting the data. Typical ways of regularization involve adding some form of magnitude measure of the weights to the loss function. CNNs take a different approach toward regularization: they exploit hierarchical patterns in the data and use smaller and simpler patterns to assemble more complex patterns. Therefore, on the scale of connectivity and complexity, CNNs are at the lower extreme. Compared to other image classification / processing algorithms, CNNs use relatively little preprocessing. This means that the network learns filters that are manually engineered in traditional algorithms. This independence from prior knowledge and human effort in feature design is a major advantage. 2.10.2 Deep Learning for Image / Video Encoding and Decoding Image / video compression based on deep learning generally has two meanings: end-to-end compression based purely on neural networks and traditional frameworks enhanced by neural networks. The first type generally adopts an autoencoder-like structure implemented by a convolutional neural network or a recurrent neural network. Although purely relying on neural networks for image / video compression can avoid any manual optimization or manual design, the compression efficiency may not be satisfactory. Therefore, the work distributed in the second type uses neural networks as an aid to enhance the traditional compression framework by replacing or enhancing some modules. In this way, they can inherit the advantages of highly optimized traditional frameworks. For example, a fully connected network for intra-frame prediction is proposed. In addition to intra-frame prediction, deep learning is also used to enhance other modules. For example, the loop filter of HEVC with a convolutional neural network is replaced, and promising results are achieved. Neural networks are applied to improve arithmetic codec engines. 2.10.3 In-loop filtering based on convolutional neural networks In lossy image / video compression, the reconstructed frame is an approximation of the original frame because the quantization process is irreversible and thus results in distortion of the reconstructed frame. To mitigate this distortion, a convolutional neural network can be trained to learn the mapping from the distorted frame to the original frame. In practice, the training must be performed before deploying the CNN-based in-loop filtering. 2.10.3.1 Training The goal of the training process is to find optimal values ​​for the parameters including weights and biases. First, a codec (such as HM, JEM, VTM, etc.) is used to compress the training dataset to generate distorted reconstructed frames. The reconstructed frame is then fed into the CNN, and the cost is calculated using the output of the CNN and the reference truth frame (original frame). Common cost functions include SAD (sum of absolute differences) and MSE (mean square error). Next, the gradient of the cost with respect to each parameter is derived through the back propagation algorithm. Using the gradient, the value of the parameter can be updated. The above process is repeated until the convergence criteria are met. After training is completed, the derived optimal parameters are saved for the inference stage. 2.10.3.2 Convolution Process During convolution, the filter is moved across the image from left to right and top to bottom, with a 1-pixel column change for horizontal movement and then a 1-pixel row change for vertical movement. The amount of movement between applications of the filter to the input image is called the stride, which is almost always symmetric across the height and width dimensions. For both height and width movement, the default stride is (1,1) in both dimensions. Fig.15A An example diagram 1500 showing the architecture of the proposed CNN filter. Fig. 15B An example diagram 1550 showing the construction of a ResBlock (residual block) in a CNN filter. In most deep convolutional neural networks, residual blocks are used as basic modules and are stacked several times to build the final network, where in one example, the residual block is obtained by combining a convolutional layer, a ReLU / PReLU activation function, and a convolutional layer, as shown in FIG. Fig. 15B shown. 2.10.3.3 Reasoning During the inference phase, the distorted reconstructed frames are fed into the CNN and processed by the CNN model, whose parameters have been determined in the training phase. The input samples of the CNN can be the reconstructed samples before or after DB, or the reconstructed samples before or after SAO, or the reconstructed samples before or after ALF. 3. Question Prior art designs of NN filters are applied to generate reconstructions. Multiple NN filters are directly selected. Therefore, NN filters cannot be adaptively fused. 4. Detailed solution The following detailed embodiments should be considered as examples to explain the general concept. These embodiments should not be interpreted in a narrow sense. In addition, these embodiments can be combined in any way. In order to solve the above problems, it is proposed to combine or fuse the filtered samples generated by the NN filter and / or the non-NN filter. The present disclosure details how to combine filters and how to utilize or control the filtered samples to generate adaptive reconstruction. In the present disclosure, an independent filter (ID-filter) means that the filter is not exactly the same as other filters, and some parts of the filter are different, such as the input of the filter, the structure of the filter, the parameters of the filter, and the neural network model of the filter. In one example, the design of the ID-filter is unique and different from the design of other filters. In one example, when the filters share a consistent structure or consistent parameters or a consistent neural network model, the input of the ID-filter is different. The ID-filter can be any kind of filter, including a filter without a neural network (non-NN filter) and a filter with a neural network (NN filter). The non-NN filter can be one of a deblocking filter (DF), a sample adaptive offset (SAO), an adaptive loop filter (ALF), etc. The NN filter can be any kind of NN filter, such as a convolutional neural network (CNN) filter. In the following discussion, the NN filter may also be referred to as a CNN filter. In the following discussion, a video unit may be a sequence, a picture, a slice, a tile, a brick, a sub-picture, a CTU / CTB, a CTU / CTB row, one or more CU / CBs, one or more CTU / CTBs, one or more VPDUs (virtual pipeline data units), a sub-region within a picture / slice / tile / brick. A parent video unit represents a unit that is larger than a video unit. Typically, a parent unit will contain several video units. For example, when a video unit is a CTU, a parent unit may be a slice, a CTU row, multiple CTUs, etc. The cross-component SAO is denoted as CCSAO. The cross-component ALF is denoted as CCALF. The bilateral in-loop filter is denoted as BIF. The deblocking filter is denoted as DB. The width and height of the video unit are denoted as W and H, respectively. Regarding the integration of ID filters during the decoding or encoding process 1. It is proposed that at least two ID filters may be included in a compatible decoder or encoder. a. In one example, an ID filter can be used to filter the reconstruction. i. In one example, a reconstruction is generated from the prediction and the residual. ii. In one example, the reconstruction is the filtered output signal of the other filter. 1) In one example, the filter may or may not be an ID filter. b. In one example, the ID filter can be a non-NN filter and a NN filter. i. In one example, the non-NN filter can be DB, SAO, BIF, ALF, CCSAO, CCALF. ii. In one example, the NN filter can be an in-loop filter based on a CNN. iii. In one example, filters may be applied according to a specific or adaptive order. c. In one example, the ID filter may be a NN filter. i. In one example, the NN models of the NN filters are different. ii. In one example, when the NN models of the NN filters are the same, the inputs of the NN models are different. 1) In one example, the QPs used for the NN filters are different. a. In one example, the model of the NN filter is different for different QPs. b. In one example, QP is an input parameter of the model of the NN filter. d. In one example, the ID filter may be a non-NN filter. i. In one example, the non-NN filters are DB and SAO. ii. In one example, the non-NN filters are DB and ALF. iii. In one example, the non-NN filters are SAO and ALF. iv. In one example, the non-NN filter is DB and / or SAO and / or BIF and / or CCSAO and / or CCALF and / or ALF. 2. ID filters can be combined to design new ID filters. a. In one example, the ID filter f A and ID filter f B can be combined into a new filter f C . i. In one example, f C can be used with other ID filters (such as f A 、f B )Different ID filters. ii. In one example, the ID filter f A and ID filter f B Can be the same filter. b. In one example, DB and SAO can be combined into a new ID filter. c. In one example, the DB and NN filters can be combined into a new ID filter. d. In one example, the SAO and NN filters can be combined into a new ID filter. e. In one example, the BIF and NN filters can be combined into a new ID filter. f. In one example, the ALF and NN filters can be combined into a new ID filter. g. In one example, the same or different NN filters can be combined into a new ID filter. h. In one example, the number of ID filters in the joint process can be N. i. In one example, N is 0, 1, 2, 3, 4, etc. ii. In one example, N is not less than 2. iii. In one example, N may be a positive integer. About the combination of ID filters during the decoding or encoding process 3. ID filters can be separated from other ID filters. a. In one example, the input information of the ID filter can be different. b. In one example, due to the ID filter, only a limited number N of filtered reconstructed signals can be retained according to the encoding or decoding rules. i. In one example, the rules may depend on the codec mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). ii. In one example, N is 0, 1, 2, 3, 4, etc. iii. In one example, N is not less than 1. iv. In one example, N may be a positive integer. c. In one example, the ID filter may be separate from other ID filters used for a portion of a video unit. d. In one example, the ID filter may be separate from other ID filters used for all video units. 4. ID filters can be combined with other ID filters. a. In one example, the input information of the ID filter can be the same. i. In one example, ID filters may be applied in parallel. ii. In one example, SAO and BIF may be applied in parallel. b. In one example, the output of the ID filter can be the input to other filters. i. In one example, the reconstruction after filtering by the ID filter can be the input information of other filters. ii. In one example, the NN filter may be applied after DB and / or SAO and / or BIF and / or CCSAO and / or CCALF and / or ALF. iii. In one example, a NN filter may be applied after another NN filter. c. In one example, the input of the ID filter can be the output of other filters. i. In one example, the reconstruction after filtering by other filters can be the input signal of the ID filter. ii. In one example, DB and / or SAO and / or BIF and / or CCSAO and / or CCALF and / or ALF may precede the NN filter. iii. In one example, a NN filter may precede another NN filter. d. In one example, skip connections can be applied to ID filters. i. In one example, the input of ID filter A and the output of ID filter A can be the input of another ID filter B. 1) In one example, the ID filter A may be a NN filter or a DB filter or a SAO. 2) In one example, the ID filter A may be a fusion of a NN filter and a DB filter. 3) In one example, ID filter B may be an ALF and / or a CCALF. ii. In one example, the input of ID filter A and the output of ID filter A can be fused by another ID filter B. 1) In one example, the ID filter A may be a NN filter or a DB filter or a SAO. 2) In one example, the ID filter A may be a fusion of a NN filter and a DB filter. 3) In one example, ID filter B may be an ALF and / or a CCALF. e. In one example, cross-component concatenation may be applied to the ID filter. i. In one example, for a component C 0 The filtering can depend on the other component C 1 In other words, C 1 ID filters can be used to guide C 0 ID filter. 1) In one example, C 0 can be the brightness component, and C 1 Can be a chroma component. 2) In one example, C 0 can be a chrominance component, and C 1 Can be a luminance component. 3) In one example, C 0 can be the Y component, and C 1 Can be Cb and / or Cr components. 4) In one example, C 0 It can be Cb component, and C 1Can be Y and / or Cr components. 5) In one example, C 0 It can be the Cr component, and C 1 Can be Y and / or Cb components. ii. In one example, the cross-component connection may depend on the codec sample, the codec mode of the video unit / Statistics (e.g., prediction mode, qp, temporal layer, slice type, etc.). 1) In one example, in C 1 Samples before and / or after the ID filter can be used for C 0 ID filter. 2) In one example, C 1 The information generated by the ID filter can be used for C 0 ID filter. a. In one example, the information may be the boundary strength in a DB filter. 3) In one example, C 0 can be the brightness component, and C 1 Can be a chroma component. 4) In one example, C 0 can be a chrominance component, and C 1 Can be a luminance component. 5) In one example, C 0 can be the Y component, and C 1 Can be Cb and / or Cr components. 6) In one example, C 0 It can be Cb component, and C 1 Can be Y and / or Cr components. iii. In one example, the ID filter may be a DB filter. iv. In one example, the ID filter may be SAO. v. In one example, the ID filter can be an ALF. vi. In one example, the ID filter may be a NN filter. f. In one example, the ID filter may be applied before other filters. i. In one example, the NN filter may be applied after DB or SAO or ALF. ii. In one example, a NN filter may be applied after another NN filter. g. In one example, the ID filter can be applied after other filters. i. In one example, DB and / or SAO and / or BIF and / or CCSAO and / or CCALF and / or ALF may precede the NN filter. ii. In one example, a NN filter may precede another NN filter. h. In one example, ID filters may be applied according to a specific or adaptive order. i. In one example, DB, NN filter, SAO and ALF are applied sequentially. i. In one example, the order in which the ID filter and / or other filters are applied may depend on the codec mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). j. In one example, whether and / or how to utilize ID filters and / or other filters may depend on the encoding / decoding mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). k. In one example, an ID filter may be combined with other ID filters for a portion of a video unit. 1. In one example, an ID filter can be combined with other ID filters for all video units. m. In one example, the combination of ID filters can be tailored. i. In one example, cropping may depend on the codec mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). ii. In one example, the cropping may depend on the bit depth of the input signal and / or the internal signal. 5. ID filters can be combined by skip connections. Note that ID filter A i The output is I i (x,y), the output of ID filter B is O(x,y). Thus, ID filter A i The input is I i-1 (x,y). a. In one example, N ID filters may be involved in the skip connection. i. In one example, N is 0, 1, 2, 3, 4, 5, 6, 7. ii. In one example, N may be a positive integer or zero. b. In one example, a skip connection may be a short connection. i. In one example, a skip connection may involve an ID filter A i . Fig.16A Shows the jump link ID filter A involved in the connection i Schematic diagram of c. In one example, the skip connection may be a long connection. i. In one example, multiple ID filters A may be involved in a jump connection i To A j . Fig. 16B It shows that multiple ID filters A are involved in the skip connection i To A j Schematic diagram of . 1) In one example, ID filter index i is less than or greater than ID filter index j. 2) In one example, NN filters and / or DB filters may be involved in the skip connection. 3) In one example, samples before the NN filter and / or DB filter may be additional inputs to the ALF and / or CCALF. d. In one example, skip connections can be applied to multiple ID filters A applied in serial order i To A j . Fig. 16C shows a case where skip connections are applied to multiple ID filters A applied in serial order i To ID filter A j Schematic diagram of the structure. i. In one example, ID filter A i The input and ID filter A j The output of can be the input of ID filter B. And ID filter A i In A j After / Before is applied. 1) In one example, ID filter A i It can be a NN filter or a DB filter. 2) In one example, ID filter A i It can be a fusion of multiple ID filters. a. In one example, ID filter A i It can be NN filter and DB filter. 3) In one example, ID filter A i Can be SAO filter and / or CCSAO. 4) In one example, ID filter B may be an ALF and / or a CCALF. e. In one example, a skip connection can be applied to an ID filter A i . i. In one example, ID filter A i The input and ID filter A iThe output of can be the input of another ID filter B. Fig.16D shows where the ID filter A i The input and ID filter A i The output of is a schematic diagram of the structure of the input of another ID filter B. 1) In one example, ID filter A i It can be a NN filter or a DB filter or a SAO. 2) In one example, ID filter A i It can be a fusion of multiple ID filters. a. In one example, ID filter A i It can be NN filter and DB filter. 3) In one example, ID filter B may be an ALF and / or a CCALF. f. In one example, the skip connection can be an addition and / or subtraction operation with weighted parameters. Fig.16E A schematic diagram showing a structure in which a skip connection is an addition and / or subtraction operation with weighted parameters. i. In one example, O(x, y) = w 1 *f 1 (I i-1 (x,y))+w 2 *f 2 (I i (x,y)). 1) In one example, f 1 (*) and / or f 2 (*) Can be a KxK filter with weighting parameters. 2) In one example, w 1 and / or 2 Can be equal to 1.0. 3) In one example, w 1 and / or 2 Can be equal to 0.0. 4) In one example, w 1 and w 2 The sum may be equal to a predefined value T. a. In one example, T may be equal to 1.0. ii. In one example, samples around (x, y) can be used in the function. g. In one example, the skip connection can be a cascade operation. h. In one example, the skip connection can be a fusion operation. i. In one example, the input of ID filter B can be I i-1 (x,y) and Ii (x,y). j. In one example, the input of ID filter B can be I i-1 (x,y) and I j (x,y). k. In one example, the output O(x,y) of ID filter B can be ID filter A i Input I i-1 (x,y) input and ID filter A i Output I i The function f(*,*) of (x,y). i. In one example, f(*,*) may be an addition and / or subtraction function with weighting parameters. ii. In one example, f(*,*) can be a fusion function. iii. In one example, f(*,*) may be a cascade function. iv. In one example, O(x, y) = f(I i-1 (x,y),I i (x,y)). v. In one example, samples around (x, y) can be used in the function. 1) In one example, the sample may be located in a predefined area, and the sample (x, y) is located at the center. Fig.17A A schematic diagram showing a sample according to an embodiment of the present disclosure is shown. a. In one example, the area may be a diamond or a rectangle or a quadrilateral. i. In one example, the size of the region may be KxK. 1. In one example, K is 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14. b. In one example, the number of samples in a region may be L. i. In one example, L is 5, 13, 25, 41, 61. 2) In one example, the parameters may be the same for symmetric or diagonal samples. 3) In one example, sample I i-1 The shape and / or number of (x,y) can be different from the sample I j The shapes and / or numbers of (x,y) are different / same. vi. In one example, ID filter A i It can be a NN filter or a DB filter or a SAO. vii. In one example, ID filter A i It can be a fusion of multiple ID filters. 1) In one example, ID filter A i It can be NN filter and DB filter. viii. In one example, ID filter B may be an ALF and / or a CCALF. ix. In one example, ID filter A i Either a 5x5 or 3x3 filter can be used. x. In one example, ID filter B can use a 7x7 or 5x5 filter. l. In one example, the output O(x,y) of ID filter B can be ID filter A i Input I i-1 (x,y) and ID filter A j Output I j The fusion of (x,y). i. In one example, f(*,*) may be an addition and / or subtraction function with weighting parameters. ii. In one example, f(*,*) can be a fusion function. iii. In one example, f(*,*) may be a cascade function. iv. In one example, O(x, y) = f(I i-1 (x,y),I j (x,y)). v. In one example, samples around (x,y) can be used in the function. 1) In one example, the sample may be located in a predefined area, and the sample (x, y) is located at the center. Fig. 17B A schematic diagram showing a sample according to an embodiment of the present disclosure is shown. a. In one example, the area may be a diamond or a rectangle or a quadrilateral. i. In one example, the size of the region may be KxK. 1. In one example, K is 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14. b. In one example, the number of samples d in a region may be L. i. In one example, L is 5, 13, 25, 41, 61. 2) In one example, the parameters may be the same for symmetric or diagonal samples. 3) In one example, sample I i-1 The shape and / or number of (x,y) can be different from the sample I j The shapes and / or numbers of (x,y) are different / same. vi. In one example, ID filter A i It can be a NN filter or a DB filter. vii. In one example, ID filter A i It can be a fusion of multiple ID filters. 1) In one example, ID filter A i It can be NN filter and DB filter. viii. In one example, ID filter A i Can be SAO filter and / or CCSAO. ix. In one example, ID filter B can be ALF and / or CCALF. m. In one example, multiple skip connection operations may be combined in any manner. i. In one example, a skip connection can be applied to ID filter A i and ID filter B, where another skip connection can be applied to ID filter A i and ID filter B. 6. Whether and / or how to utilize the skip connection of the ID filter may depend on the coding mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). a. In one example, it may depend on prediction mode, qp, temporal layer, slice type, etc. b. In one example, it may depend on the quantization step size. c. In one example, it may depend on the time layer. d. In one example, it may depend on the stripe type. e. In one example, it may depend on the block size of the video unit. f. In one example, it may depend on the color component. g. In one example, it may depend on a signal in the bitstream. i. In one example, the signal may be in a sequence and / or a picture and / or a slice and / or a tile and / or a brick and / or a sub-picture and / or a CTU / CTB and / or a CTU / CTB row and / or a CU / CB and / or multiple CU / CBs. 7. ID filter can be used more than once. a. In one example, filters with the same ID can be connected sequentially. i. In one example, the number of ID filters may be N. 1) In one example, N is 0, 1, 2, 3, 4. 2) In one example, N is not less than 2. 3) In one example, N may be a positive integer. b. In one example, an ID filter f A Respectively with other filters f B and filter f C combination. 8. Any ID filter can be an internal stage or part of the filtering process. a. In one example, the filtered samples are attributable to a filtering process. i. In one example, the filtered samples may be placed into a decoded picture buffer. ii. In one example, the filtered samples may be the final display signal. b. In one example, the filtering process may be a combination, fusion or selection of multiple groups of ID filters. i. In one example, there are one or more ID filters in a set of ID filters. About the use of ID filter 9. Whether and / or how to utilize the ID filter may depend on the codec mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). a. In one example, it may depend on prediction mode, qp, temporal layer, slice type, etc. b. In one example, it may depend on the quantization step size. c. In one example, it may depend on the time layer. d. In one example, it may depend on the stripe type. e. In one example, it may depend on the block size of the video unit. f. In one example, it may depend on the color component. g. In one example, it may depend on a signal in the bitstream. i. In one example, the signal may be in a sequence and / or a picture and / or a slice and / or a tile and / or a brick and / or a sub-picture and / or a CTU / CTB and / or a CTU / CTB row and / or a CU / CB and / or multiple CU / CBs. h. In one example, the number of ID filters may be N. i. In one example, N is 0, 1, 2, 3, 4, 5, 6. ii. In one example, N may depend on the statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). 1) In one example, the use of ID filters may depend on the sequence and / or picture and / or slice and / or tile and / or brick and / or sub-picture and / or CTU / CTB and / or CTU / CTB row and / or a CU / CB and / or or indicators in multiple CUs / CBs. i. In one example, the ID filter can be a group of filters. j. In one example, an ID filter can be used in the fusion process. 10. The above method can be applied to any kind of NN-based encoding and decoding methods. a. In one example, the ID filter can be replaced by an intra prediction / improvement method. i. In one example, it can be a NN-based method and / or a non-NN-based method. b. In one example, the ID filter may be alternatively passed through an inter-frame prediction / improvement method. ii. In one example, it can be a NN-based method and / or a non-NN-based method. c. In one example, they can be applied in a unified NN filtering approach (in-loop or post-processing). d. In one example, they can be applied to a non-uniform NN filtering approach (in-loop or post-processing). e. In one example, they can be applied to NN based intra and / or inter methods. f. In one example, they can be applied to non-NN based intra and / or inter methods. g. In one example, one of the ID filters may be a NN-based intra / inter method, and the other of the ID filters may be a non-NN-based intra / inter method. i. In one example, they can be fused by the above method. 5. Other aspects About the Fusion of ID Filters 11. ID filters can be fused to generate samples based on the filtered reconstruction generated by the ID filters. a. Whether and / or how the fusion ID filter can be adaptive. b. Whether and / or how to blend ID filters may depend on the statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). c. Whether and / or how to fuse ID filters may depend on equations and / or models. i. In one example, the model can be a neural network model. ii. In one example, the model may be a linear model. d. In one example, the fused samples may be placed into a decoded picture buffer. e. In one example, the fused sample can be the final display signal. f. In one example, the ID filter can be any type of ID filter. i. In one example, an ID filter may be a NN filter and / or a non-NN filter. g. In one example, due to the fusion of ID filters, clipping can be applied to the samples. i. In one example, cropping may depend on the codec mode / statistics of the video unit (e.g., prediction mode, qp, time layer, slice type, etc.). ii. In one example, the cropping may depend on the bit depth of the input signal and / or the internal signal. 12. Whether and / or how to fuse ID filters may depend on equations and / or models. a. In one example, a linear function may be applied to the current filtered sample with the ID filter. i. In one example, neighboring samples may be involved in a linear function. 1) In one example, neighboring samples may include adjacent and / or non-adjacent samples. b. In one example, the fused sample y in the video unit can be obtained by the linear model y = ∑a k *x k +b to derive. Here, x k is due to the ID filter f k The filtered samples, k is the ID filter f k The index of a, k ranges from 1 to K. And K, a k and b are parameters. i. In one example, the fused sample y can be the final reconstruction. 1) In one example, the cropped value of the fused sample y can be the final reconstruction. ii. In one example, b may be equal to zero, and y = ∑a k *x k . iii. In one example, ∑a k Can be equal to a constant value, such as 1.0, 0.0. 1) In one example, ∑a k =1 and y = ∑a k-1 *x k-1 +(1-∑a k-1 )*x k +b. iv. In one example, b = 0 and ∑a k =1 and y = ∑a k-1 *x k-1 +(1-∑a k-1 )*x k . v. In one example, K can be 1, 2, 3, 4, 5. 1) In one example, K = 1 and y = a 1 *x 1 +b. 2) In one example, K = 2 and y = a 1 *x 1 +a 2 *x 2 +b 3) In one example, K is 3 and y=a 1 *x 1 +a 2 *x 2 +a 3 *x 3 +b vi. In one example, K=2 and b=0, so that y=a 1 *x 1 +a 2 *x 2 vii. In one example, K = 2 and ∑a k =1, so that y = a 1 *x 1 +(1-a 1 )*x 2 +b. viii. In one example, K=2 and b=0 and Σa k =1, so that y = a 1 *x 1 +(1-a 1 )*x 2 . ix. In one example, K=3 and b=0, so that y=a 1 *x 1 +a 2 *x 2 +a 3 *x 3 xi. In one example, K=3 and b=0 and ∑a k =1, so that y = a 1 *x 1 +a 2 *x 2 +(1-a 1 -a2 )*x 3 xii. In one example, when K=2. 1) In one example, the ID filter f 1 and f 2 is the NN filter. a. In one example, the ID filter f 1 The model or network structure can be combined with the ID filter f 2 The models are different. b. In one example, the ID filter f 1 The model or network structure can be an ID filter f 2 A simplified version of the model. c. In one example, the ID filter f 1 The model can be the ID filter f 2 A simplified version of the model. d. In one example, the ID filter f 1 The input parameters can be compared with the ID filter f 2 The input parameters are different. i. In one example, the ID filter f 1 The model or network structure can be combined with the ID filter f 2 The model is the same. ii. In one example, the ID filter f 1 The model or network structure can be combined with the ID filter f 2 The models are different. 1. In one example, f 1 and f 2 The difference between may be training data, such as a quality-level indicator. iii. In one example, the quality-level indicator is used as input to the ID filter f 1 and f 2 Can be different. iv. In one example, the above quality-level indicator may be QP or λ or a constant rate factor (CRF) value or a bit rate. 2) In one example, the ID filter f 1 is a NN filter, and f 2 It is DB or SAO or BIF or CCSAO or CCALF or ALF. 3) In one example, the ID filter f 1 is a NN filter, and f 2 It is a set of DB and / or SAO and / or BIF and / or CCSAO and / or CCALF and / or ALF. xiii. In one example, when K=3. 1) In one example, the ID filter f 1 、f 2 and f 3 is the NN filter. a. In one example, the ID filter f 1 and / or f 2 and / or f 3 The models or network structures can be different from each other. b. In one example, the ID filter f 1 and / or f 2 The model or network structure can be an ID filter f 3 A simplified version of the model. c. In one example, the ID filter f 1 and / or f 3 The model or network structure can be an ID filter f 2 A simplified version of the model. d. In one example, the ID filter f 2 and / or f 3 The model or network structure can be an ID filter f 1 A simplified version of the model or network structure. e. In one example, the ID filter f 1 and / or f 2 and / or f 3 The input parameters can be different from each other. i. In one example, the ID filter f 1 and / or f 2 and / or f 3 The models or network structures can be the same as each other. ii. In one example, the ID filter f 1 and / or f 2 and / or f 3 The models or network structures can be different from each other. 1. In one example, f 1 and / or f 2 and / or f 3 The difference between may be training data, such as a quality-level indicator. iii. In one example, the quality-level indicator is used as input to the ID filter f 1 and / or f 2 and / or f 3 Can be different. iv. In one example, the above quality-level indicator may be QP or λ or a constant rate factor (CRF) value or a bit rate. 2) In one example, the ID filter f 1 and / or f 2 and / or f 3 It can be NN filter or DB or SAO or BIF or CCSAO or CCALF or ALF. 3) In one example, the ID filter f 1 and / or f 2 and / or f 3 Can be a NN filter or a set of DB and / or Or SAO and / or BIF and / or CCSAO and / or CCALF and / or ALF. xiv. In one example, K and / or a k and / or b may be adaptive or constant values ​​or indicated by one or more indicators. xv. In one example, K and / or a k and / or b may depend on the coding mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). xvi. In one example, K and / or a k and / or b can be pre-designed values. 1) In one example, a k It can be 1.0, 7 / 8, 3 / 4, 0.75, 1 / 2, 0.5, 1 / 4, 0.25, 0.0. xvii. In one example, K and / or a k and / or b may be selected from pre-designed values ​​and / or constant values ​​and / or indicators. c. In one example, a non-linear function may be applied to the current filtered sample and / or neighboring samples (including adjacent or non-adjacent) with the ID filter. 13. Whether and / or how to construct the candidate list for the fusion of ID filters may depend on the codec statistics of the video unit (eg, prediction mode, qp, slice type, etc.). a. Number of candidate lists N C Can be a constant. i. In one example, N C It can be 0, 1, 2, 3, 4. ii. In one example, N C Can be a positive integer. b. Number of candidate lists N C It can depend on the number N of NN filters f . i. In one example, N C Can be equal to N f . ii. In one example, N C Can be equal to N f +M. 1) In one example, M can be -1, 0, 1, 2, 3, 4. 2) In one example, M may be a positive integer. iii. In one example, N C Can be equal to 2^N f . c. Number of candidate lists N C May be indicated by one or more indicators. d. A fusion candidate can include N k ID filter. i. In one example, N k It can be 0, 1, 2, 3, 4. ii. In one example, N k It can depend on the number N of NN filters f . 1) In one example, N k Can be equal to N f . iii. In one example, N k can be equal to K disclosed above in the linear model. e. In one example, there may be 3 NN filters f A 、f B and f C . i. In one example, the first fusion candidate mode may be F(f A ) and F(f B )’s fusion. ii. In one example, the second fusion candidate mode may be F(f B ) and F(f C )’s fusion. iii. In one example, the third fusion candidate mode may be F(f C ) and F(f A )’s fusion. iv. In one example, the above F(f X ) can be combined with f X same. v. In one example, the above F(f X ) can be f X A simplified version of . vi. In one example, F(f disclosed above X ) can depend on f X . vii. In one example, the above F(f X ) for different f X Can be adaptive. f. In one example, the candidate order may be adaptive. g. In one example, the candidate order may depend on the codec statistics of the video unit (eg, prediction mode, qp, slice type, etc.). h. In one example, the candidate order can be pre-designed. i. In one example, one or any of the fused candidates may include only NN filters. j. In one example, one or any of the fused candidates may include a NN filter and a non-NN filter. i. In one example, the non-NN filter can be DB or SAO or BIF or CCSAO or CCALF or ALF. ii. In one example, the non-NN filter may be a set of DB and / or SAO and / or BIF and / or CCSAO and / or CCALF and / or ALF. About the use of ID filter fusion results 14. Whether and / or how to utilize the fusion result of the ID filter may depend on the encoding / decoding mode / statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). a. In one example, it may depend on prediction mode, qp, temporal layer, slice type, etc. b. In one example, it may depend on the quantization step size. c. In one example, it may depend on the time layer. d. In one example, it may depend on the stripe type. e. In one example, it may depend on the block size of the video unit. f. In one example, it may depend on the color component. g. In one example, it may depend on a signal in the bitstream. i. In one example, the signal may be in a sequence and / or a picture and / or a slice and / or a tile and / or a brick and / or a sub-picture and / or a CTU / CTB and / or a CTU / CTB row and / or a CU / CB and / or multiple CU / CBs. h. In one example, the number of ID filters may be N. i. In one example, N is 0, 1, 2, 3, 4, 5, 6. ii. In one example, N may depend on the statistics of the video unit (eg, prediction mode, qp, temporal layer, slice type, etc.). 1) In one example, the use of ID filters may depend on the sequence and / or picture and / or slice and / or tile and / or brick and / or sub-picture and / or CTU / CTB and / or CTU / CTB row and / or a CU / CB and / or or indicators in multiple CUs / CBs. i. In one example, the ID filter can be one of a group of filters. j. In one example, clipping may be applied to samples as a result of the fusion of the ID filters. i. In one example, cropping may depend on the codec mode / statistics of the video unit (e.g., prediction mode, qp, time layer, slice type, etc.). ii. In one example, the cropping may depend on the bit depth of the input signal and / or the internal signal. k. In one example, the fused result may be placed into a decoded picture buffer. 1. In one example, the fusion result may be a final display signal. On the simplification of NN filter model in the fusion process 15. The model of the ID filter in the fusion process can be a simplified version of the models of other ID filters in the fusion process. a. In one example, the ID filter may be a NN filter. i. In one example, the depth of the NN filter model can be different. 1) In one example, the NN filter model used in the fusion process can have a shallower depth. ii. In one example, the feature maps of the NN filter model may be different. 1) In one example, the NN filter model used in the fusion process can have fewer feature maps. iii. In one example, the number of ResBlocks of the NN filter model may be different. 1) In one example, the number of ResBlocks of the NN filter model used in the fusion process can be smaller. 2) In one example, the number of ResBlocks is 1, 2, 3, 4, 5, 6. iv. In one example, the convolution kernels of the NN filter model can be different. v. In one example, both the simplified model and the normal model of the ID filter can be used in the fusion process. Signaling of Fusion Parameters 16. The use / enabling of fusion of ID filters may be controlled by adding one or more syntax elements at a first level (eg, sequence level, such as in an SPS or sequence header). a. One or more syntax elements in the first level may include a first flag indicating whether the fusion process is enabled. i. In one example, if the flag is true or 1, the fusion process is enabled. ii. In one example, if the flag is false or 0, the fusion process is not enabled. b. The one or more syntax elements in the first level may include a syntax element indicating the number of fusion modes that can be used. i. In one example, the syntax element may be signaled only if fusion is enabled. c. The one or more syntax elements in the first level may include a syntax element indicating whether a fusion process is used in the first level. d. The one or more syntax elements in the first level may include a syntax element indicating whether the fusion process can be adaptively / non-adaptively used or selected in the second level. 17. Alternatively, one or more syntax elements at the second level (eg, picture level, such as picture header (PH) or PPS; or slice level, such as slice header (SH)) may be further signaled. In the following discussion, PH may be replaced by SH. a. Syntax elements at the second level may be conditionally signaled. i. In one example, whether to signal may be based on what is signaled in the first level. b. The one or more syntax elements in the second level may include a first flag indicating whether a fusion process is enabled in the second level. c. The one or more syntax elements in the second level may include a syntax element indicating the number of fusion modes that can be used in the second level. i. In one example, the number of fusion modes may be a default value. 1) In one example, the default value can be 0, 1, 2, 3. ii. In one example, these may not be signaled. d. The one or more syntax elements in the second level may include a syntax element indicating whether a fusion process is used in the second level. e. The one or more syntax elements in the second level may include a syntax element indicating an index of a fusion candidate. i. In one example, the index of the fusion candidate may be the same as the indicator of the index of the selection filter. ii. In one example, whether to signal may be based on those indicating the number of fusion modes. f. The one or more syntax elements in the second level may include syntax elements indicating the fusion parameters disclosed above. i. One or more syntax elements may indicate the number K of filters used in a fusion mode. ii. One or more syntax elements may indicate a k Whether it is indicated by a direct syntax element or an indirect syntax element (eg, an index of a pre-designed value), it is called ADP-SE. iii. One or more syntax elements (SE) may indicate parameter a k . 1) One or more syntax elements may indicate a k function. a. In one example, a k It can be equal to SE multiplied / divided by the factor T. i. In one example, T may be equal to 2^W. 1. In one example, W may be a positive integer. a. In one example, W can be 0, 2, 4, 6, 8, 10, 12, 14, 16. ii. In one example, T may be a positive integer. 2) In one example, whether to signal may depend on the previous ADP-SE. iv. One or more syntax elements may indicate parameter a k The index of . 1) In one example, it may be indicated that a k An index into several pre-designed values ​​of . 2) In one example, whether to signal may depend on the previous ADP-SE. v. One or more syntax elements may indicate the offset parameter b. vi. In one example, the syntax element of each candidate may be different. g. The one or more syntax elements in the second base may include a syntax element indicating whether the fusion process is adaptively / non-adaptively used or selected in the third level. 18. Alternatively, one or more syntax elements at a third level (eg, patch / CTU / CTB / block / sub-picture / tile / slice / region containing multiple samples) may be further signaled. a. Syntax elements at the third level may be conditionally signaled. i. In one example, whether to signal may be based on what is signaled in the first or second level. b. The one or more syntax elements in the third level may include a first flag indicating whether a fusion process is enabled in the third level. c. The one or more syntax elements in the third level may include a syntax element indicating whether a fusion process is to be used in the third level. d. The one or more syntax elements in the third level may include a syntax element indicating an index of a merge candidate. i. In one example, the index of the fusion candidate may be the same as the indicator of the index of the selection filter. ii. In one example, whether to signal may be based on those indicating the number of fusion modes in the second level. iii. In one example, whether to signal may be based on the number of indices indicating the merge candidates in the second level. e. The one or more syntax elements in the third level may include syntax elements indicating the fusion parameters disclosed above. i. One or more syntax elements may indicate whether to use parameters in the second level or the third level. ii. The following syntax elements at the third base may be conditionally signaled. 1) In one example, whether to signal may depend on those that indicate whether to use parameters in the second level or the third level. iii. One or more syntax elements may indicate the number K of filters used in a fusion mode. iv. One or more syntax elements may indicate a k Whether it is indicated by a direct syntax element or an indirect syntax element (eg, an index of a pre-designed value), it is called ADP-SE. v. One or more syntax elements (SE) may indicate parameter a k . 1) One or more syntax elements may indicate a k function. a. In one example, a k It can be equal to SE multiplied / divided by the factor T. i. In one example, T may be equal to 2^W. 1. In one example, W may be a positive integer. a. In one example, W can be 0, 2, 4, 6, 8, 10, 12, 14, 16. ii. In one example, T may be a positive integer. 2) In one example, whether to signal may depend on the previous ADP-SE. vi. One or more syntax elements may indicate parameter a k The index of . 1) In one example, it may be indicated that a k An index into several pre-designed values ​​of . 2) In one example, whether to signal may depend on the previous ADP-SE. vii. One or more syntax elements may indicate the offset parameter b. f. In one example, the syntax elements used for each candidate may be different. 19. In one example, any of the syntax elements disclosed above may be set to a default value. a. In one example, syntax elements may not be signaled. b. In one example, the default values ​​may be -1, 0, 1, 2, 3, 4. c. In one example, the syntax elements disclosed above are set to default values ​​only when the syntax elements are not signaled. i. In one example, the default value can be -1, 0, 1, 2, 3, 4. 20. In one example, the syntax elements disclosed above may be signaled through context coding. 21. In one example, the syntax elements disclosed above may be signaled via bypass codec. 22. In one example, the syntax elements disclosed above may be binarized using fixed length codec or unary codec or truncated unary codec or signed unary codec or signed truncated unary codec or truncated binary codec or k-th exponential Golomb codec or any other binary codec method. a.1) In one example, k may be a positive integer. i. In one example, k can be equal to 0, 1, 2, 3, 4, 5, 6. 23. In one example, the syntax elements disclosed above may be signaled individually for different color components. a. In one example, the syntax elements disclosed above may be signaled only for the C color component. i. In one example, C can be the Y or Cb or Cr color component. ii. In one example, C can be the R or G or B color component. iii. In one example, information of other components may be indicated by syntax elements of the X color component. b. In one example, syntax elements may be signaled for all available color components. c. In one example, syntax elements may be signaled for the luma color component. d. In one example, syntax elements may be signaled for chroma color components. i. In one example, Cb and Cr may share the same syntax elements. e. In one example, syntax elements may be signaled for Y and / or Cb and / or Cr color components. f. In one example, syntax elements may be signaled for R and / or G and / or B color components. g. In one example, brightness may indicate the Y component. h. In one example, chrominance may indicate a Cb or Cr component. b. In one example, syntax elements may be indexed by color component. 24. The above method can be applied to any kind of NN-based encoding and decoding method. h. In one example, the ID filter can be replaced by an intra prediction / improvement method. i. In one example, it can be a NN-based method and / or a non-NN-based method. i. In one example, the ID filter can be alternatively passed through an inter-frame prediction / improvement method. ii. In one example, it can be a NN-based method and / or a non-NN-based method. j. In one example, they can be applied to a unified NN filtering approach (in-loop or post-processing). k. In one example, they can be applied to a non-uniform NN filtering approach (in-loop or post-processing). 1. In one example, they can be applied to NN based intra and / or inter methods. m. In one example, they can be applied to non-NN based intra and / or inter methods. n. In one example, one of the ID filters may be a NN-based intra / inter method, and the other of the ID filters may be a non-NN-based intra / inter method. i. In one example, they can be fused by the above method. 6. Examples 6.1 Example #1 In the proposed method, the number of convolutional neural network based in-loop filters for the stripes is three. The number of filters in the fusion process is 2. The fusion equation is y = a1 *x 1 +(1-a 1 )*x 2 . First, a syntax element is signaled to indicate whether to fuse the NN filters. Second, a syntax element is signaled to indicate whether to use pre-designed parameters or directly signal a when fusion is enabled. 1 After that, the syntax element is signaled to directly signal a 1 In addition, when pre-designed parameters are used, a syntax element is signaled to indicate the index of the pre-designed parameter.

[0076] As used herein, the term "video unit" or "video block" may be a sequence, a picture, a slice, a tile, a brick, a sub-picture, a codec tree unit (CTU) / codec tree block (CTB), a CTU / CTB row, one or more codec units (CU) / codec blocks (CB), one or more CTU / CTBs, one or more virtual pipeline data units (VPDU), a sub-region within a picture / slice / tile / brick. As used herein, the term "independent filter (ID) filter" may refer to a filter that is not exactly the same as other filters, and some parts of the filter are different, such as the input of the filter, the structure of the filter, the parameters of the filter, and the neural network model of the filter. In one example, the design of the ID-filter is unique and different from the design of other filters. In one example, when the filters share a consistent structure or consistent parameters or a consistent neural network model, the input of the ID-filter is different. The ID-filter may be any kind of filter, including a filter without a neural network (non-NN filter) and a filter with a neural network (NN filter). The non-NN filter may be one of a deblocking filter (DF), a sample adaptive offset (SAO), an adaptive loop filter (ALF), and the like. The NN filter may be any kind of NN filter, such as a convolutional neural network (CNN) filter. In the following discussion, the NN filter may also be referred to as a CNN filter.

[0077] Fig.18 A flowchart of a method 1800 for video processing according to an embodiment of the present disclosure is shown. The method 1600 is implemented for conversion between a video unit of a video and a bit stream of the video.

[0078] At block 1810, a connection is applied to a plurality of filters for conversion between a video unit of a video and a bitstream of the video. In some embodiments, the connection may be a skip connection. Alternatively, the connection may be a cross-component connection.

[0079] At box 1820, multiple filters with connections are combined and applied to the video unit. For example, at least two ID filters may be included in a compatible decoder or encoder. The input information or output information of multiple filters may be associated with each other. In some embodiments, multiple filters may use the same input information. For example, multiple filters may be applied in parallel. Alternatively, multiple filters may be arranged in series, and the input information and output information of a filter in the multiple filters may be used as the input information of another filter in the multiple filters. In some other embodiments, the output information of multiple filters may be selected for further processing. Note that multiple filters may include any appropriate number of filters.

[0080] At block 1830, a conversion is performed based on the filtered video unit. In some embodiments, the conversion may include encoding the video unit into a bitstream. Alternatively or additionally, the conversion may include decoding the video unit from the bitstream. Compared to conventional solutions that directly select filters, filters can be adaptively combined for video units. In this way, codec effectiveness and codec efficiency can be improved.

[0081] In some embodiments, applying a connection to multiple filters may include: applying a jump connection to multiple filters. In some embodiments, the multiple filters include a first filter and a second filter. In this case, the input of the first filter and the output of the first filter may be the input of the second filter. Alternatively, the input of the first filter and the output of the first filter may be combined by the second filter.

[0082] In some embodiments, the first filter may be one of the following: a neural network (NN) filter, a deblocking filter, or a sample adaptive offset (SAO) filter. In some embodiments, the first filter may be a combination of a NN filter and a deblocking filter. In some embodiments, the second filter may include at least one of the following: an adaptive loop filter (ALF), or a cross-component ALF (CCALF).

[0083] In some embodiments, applying the connection to the plurality of filters may include applying the cross-component connection to the plurality of filters. In some embodiments, filtering the first component is dependent on filtering the second component, or wherein the second filter filtering the second component is used to guide the first filter filtering the first component. In one example, for a component C 0 The filtering can depend on the other component C 1 In other words, C 1 ID filters can be used to guide C 0 ID filter.

[0084] In some embodiments, the first component is a luminance component and the second component is a chrominance component. Alternatively, the first component is a chrominance component and the second component is a luminance component. In some embodiments, the first component is a Y component and the second component is at least one of the following: a Cb component or a Cr component. In some embodiments, the first component is a Cb component and the second component is at least one of the following: a Y component or a Cr component. In some embodiments, the first component is a Cr component and the second component is at least one of the following: a Y component or a Cb component.

[0085] In some embodiments, the cross-component connection depends on at least one of: a codec sample of the video unit, a codec mode of the video unit, or codec statistics of the video unit. In some embodiments, the codec statistics include at least one of: a prediction mode, a quantization parameter (QP), a temporal layer, or a slice type.

[0086] In some embodiments, samples before the second filter are used for the first filter. Alternatively or additionally, samples after the second filter are used for the first filter. 1 Samples before and / or after the ID filter can be used for C 0 ID filter.

[0087] In some embodiments, information generated by the second filter is used in the first filter. In some embodiments, the information is a boundary strength in a deblocking filter.

[0088] In some embodiments, the first component is a luminance component and the second component is a chrominance component. Alternatively, the first component is a chrominance component and the second component is a luminance component. In some embodiments, the first component is a Y component and the second component is at least one of the following: a Cb component or a Cr component. Alternatively, the first component is a Cb component and the second component is at least one of the following: a Y component or a Cr component.

[0089] In some embodiments, at least one filter of the plurality of filters is one of: a deblocking filter, a sample adaptive offset (SAO) filter, an ALF, or a NN filter.

[0090] In some embodiments, applying connections to multiple filters may include combining multiple filters via skip connections. For example, ID filters may be combined via skip connections. Note that ID filter A i The output is I i (x, y), and the output of ID filter B is O(x, y). Thus, ID filter A i The input is I i-1 (x,y).

[0091] In some embodiments, multiple filters are involved in the jump connection. In some embodiments, the number of filters involved in the jump connection is a positive integer or zero. In some embodiments, the number of filters involved in the jump connection is 0, 1, 2, 3, 4, 5, 6 or 7.

[0092] In some embodiments, the skip connection is a short connection. Fig.16A As shown, a filter is involved in the skip connection.

[0093] In some other embodiments, the skip connection is a long connection. In some embodiments, a filter set with an index from i to j is involved in the skip connection, where i and j represent filter indices and are integers. In one example, Fig. 16B As shown, multiple ID filters A may be involved in the jump connection. i To A j In some embodiments, i is less than or greater than j. In one example, ID filter index i is less than or greater than ID filter index j.

[0094] In some embodiments, at least one of the following is involved in the skip connection: a NN filter or a deblocking filter. In some embodiments, samples preceding at least one of the NN filter or the deblocking filter are additional inputs to at least one of the ALF or the CCALF.

[0095] In some embodiments, skip connections are applied to filter sets with indices from i to j, which are applied in serial order. Fig. 16C As shown, skip connections can be applied to multiple ID filters A i To ID filter A j , these ID filters are applied in serial order.

[0096] In some embodiments, the input of the first filter and the output of the second filter are inputs to the third filter, where the first filter is applied before or after the second filter. i The input and ID filter A j The output of can be the input of ID filter B. ID filter A i ID filter A j After / Before is applied.

[0097] In some embodiments, the first filter is a NN filter or a deblocking filter. In some embodiments, the first filter is a combination of a filter set. In one example, ID filter A iIt can be a NN filter or a DB filter. In some embodiments, the first filter is a combination of a NN filter and a deblocking filter. In one example, the ID filter A i It can be a fusion of multiple ID filters. In one example, ID filter A i It can be a NN filter and a DB filter. In some embodiments, the second filter is at least one of the following: a SAO filter or a cross-component SAO (CCSAO) filter. In one example, the ID filter A i Can be SAO filter and / or CCSAO.

[0098] In some embodiments, the third filter is at least one of: ALF or CCALF. In one example, ID filter B may be ALF and / or CCALF.

[0099] In some embodiments, a skip connection is applied to the first filter. In one example, a skip connection can be applied to an ID filter A. i .

[0100] In some embodiments, the input of the first filter and the output of the second filter are inputs to the third filter. Fig.16D As shown, ID filter A i The input and ID filter A i The output of can be the input of another ID filter B.

[0101] In some embodiments, the first filter is one of the following: an NN filter, a deblocking filter, or a SAO filter. i It can be a NN filter or a DB filter or a SAO. In some embodiments, the first filter is a combination of a filter set. In one example, ID filter A i It can be a fusion of multiple ID filters. In some embodiments, the first filter is a combination of an NN filter and a deblocking filter. In one example, the ID filter A i It can be a NN filter and a DB filter. In some embodiments, the third filter is at least one of ALF or CCALF. In one example, the ID filter B can be ALF and / or CCALF.

[0102] In some embodiments, the skip connection can be at least one of the following: an addition operation or a subtraction operation with a weighted parameter. Fig.16E As shown, the skip connections can be addition and / or subtraction operations with weighted parameters.

[0103] In some embodiments, O(x, y) = w 1*f 1 (I i-1 (x,y))+w 2 *f 2 (I i (x,y)), and where I i (x,y) represents the output of the filter with index i, I i-1 (x,y) represents the input of the filter with index i, O(x,y) represents the output of the third filter, and w 1 and w 2 They represent weighting parameters, and f 1 and f 2 Represent functions respectively.

[0104] In some embodiments, f 1 (*) or f 2 At least one of (*) is a KxK filter with weighting parameters, where K is an integer. In one example, w 1 and / or 2 can be equal to 1.0. In one example, w 1 and / or 2 Can be equal to 0.0. In some embodiments, w 1 and w 2 The sum may be equal to a predefined value. In some embodiments, the predefined value is equal to 1.

[0105] In some embodiments, samples around (x, y) are used in the function. In some embodiments, the skip connection is a cascade operation. In some embodiments, the skip connection is a fusion operation.

[0106] In some embodiments, the input to the third filter is I j-1 (x,y) and I j (x,y), and where I j (x,y) represents the output of the filter with index j, I j-1 (x, y) represents the input of the filter with index j. In one example, the input of ID filter B can be I i-1 (x,y) and I j (x,y).

[0107] In some embodiments, the output of the third filter is the input I of the filter with index i i-1 (x,y) and the output I of the filter with index i i In one example, the output O(x,y) of ID filter B can be the function f(*,*) of ID filter A. i Input I i-1 (x,y) and ID filter Ai Output I i In some embodiments, the output of the third filter is the input I of the filter with index i. i-1 (x,y) and the output I of the filter with index i i The function f(*,*) of (x,y).

[0108] In some embodiments, f(*, *) is at least one of the following: an addition function or a subtraction function with a weighted parameter. In some embodiments, f(*, *) may be a fusion function. Alternatively, f(*, *) is a cascade function.

[0109] In some embodiments, O(x, y)=f(I i-1 (x,y),I i (x, y)). In some embodiments, samples around (x, y) are used in the function. In some embodiments, the samples are located at a predefined area, and the sample (x, y) is located at the center of the predefined area. In some embodiments, the predefined area is one of the following: a diamond, a rectangle, or a quadrilateral. In some embodiments, the size of the predefined area is K x K, where K is an integer. In some embodiments, K is one of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14. Fig.17A A schematic diagram showing samples in a predefined area.

[0110] In some embodiments, the number of samples in the predefined area is L, where L is an integer. In some embodiments, L is one of 5, 13, 25, 41, 61. In some embodiments, the parameters of symmetric or diagonal samples are the same.

[0111] In some embodiments, sample I i-1 The shape of (x,y) and sample I j The shape of (x, y) is different. In some other embodiments, sample I i-1 The shape of (x,y) and sample I j The shapes of (x, y) are the same. Alternatively, sample I i-1 The number of (x,y) and sample I j The number of (x, y) is different. In some other embodiments, sample I i-1 The number of (x,y) and sample I j The number of (x,y) is the same.

[0112] In some embodiments, the filter with index i is one of the following: a NN filter, a deblocking filter, or a SAO filter. iIt can be a NN filter or a DB filter or a SAO.

[0113] In some embodiments, the filter with index i is a combination of a filter set. In one example, ID filter A i It can be a fusion of multiple ID filters.

[0114] In some embodiments, the filter with index i is a combination of an NN filter and a deblocking filter. i It can be NN filter and DB filter.

[0115] In some embodiments, the third filter is at least one of ALF or CCALF.In one example, ID filter B may be ALF and / or CCALF.

[0116] In some embodiments, the filter with index i uses a 5x5 or 3x3 filter. i Either a 5x5 or 3x3 filter can be used.

[0117] In some embodiments, the third filter uses a 7x7 or 5x5 filter. In one example, ID filter B can use a 7x7 or 5x5 filter.

[0118] In some embodiments, the output of the third filter is the input I of the filter with index i i-1 (x,y) and the output I of the filter with index j j In one example, the output O(x,y) of ID filter B can be the function f(*,*) of ID filter A. i Input I i-1 (x,y) and ID filter A i Output I j The fusion of (x,y).

[0119] In some embodiments, f(*, *) is at least one of the following: an addition function or a subtraction function with weighted parameters. In some embodiments, f(*, *) is a fusion function. Alternatively, f(*, *) is a cascade function. In some embodiments, O(x, y) = f(I i-1 (x,y),I j (x,y)). In some embodiments, samples around (x,y) are used in the function.

[0120] In some embodiments, the sample is located at a predefined area, and the sample (x, y) is located at a center position of the predefined area. In some embodiments, the predefined area is one of the following: a diamond, a rectangle, or a quadrilateral.

[0121] In some embodiments, the size of the predefined area is K x K, where K is an integer. In some embodiments, K is one of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14. Fig. 17B A schematic diagram showing samples in a predefined area.

[0122] In some embodiments, the number of samples in the predefined area is L, where L is an integer. In some embodiments, L is one of 5, 13, 25, 41, and 61.

[0123] In some embodiments, the parameters of symmetrical or diagonal samples are the same. In some embodiments, sample I i-1 The shape of (x,y) and sample I j The shapes of (x, y) are different. In some embodiments, sample I i-1 The shape of (x,y) and sample I j The shapes of (x, y) are the same. Alternatively, sample I i-1 The number of (x,y) and sample I j The number of (x, y) is different. In some embodiments, sample I i -1 The number of (x,y) and sample I j The number of (x,y) is the same.

[0124] In some embodiments, the filter with index i is one of the following: a NN filter, a deblocking filter, or a SAO filter. i It can be a NN filter or a DB filter.

[0125] In some embodiments, the filter with index i is a combination of a filter set. In one example, ID filter A i It can be a fusion of multiple ID filters. In some embodiments, the filter with index i is a combination of an NN filter and a deblocking filter. In one example, ID filter A i It can be NN filter and DB filter.

[0126] In some embodiments, the filter with index i is at least one of the following: a SAO filter or a CCSAO filter. i Can be SAO filter and / or CCSAO.

[0127] In some embodiments, the third filter is at least one of: ALF or CCALF. In one example, ID filter B may be ALF and / or CCALF.

[0128] In some embodiments, multiple skip connection operations may be combined. In one example, multiple skip connection operations may be combined in any manner.

[0129] In some embodiments, a skip connection is applied to the filter with index i and the third filter, and another skip connection is applied to the filter with index j and the third filter. In one example, a skip connection may be applied to ID filter A. i and ID filter B, where another skip connection can be applied to ID filter A j and ID filter B.

[0130] In some embodiments, whether to utilize the skip connection of multiple filters and / or the method of utilizing the skip connection of multiple filters depends on at least one of the following: the codec sample of the video unit, the codec mode of the video unit, or the codec statistics of the video unit. In some embodiments, whether to utilize the skip connection of multiple filters and / or the method of utilizing the skip connection of multiple filters depends on at least one of the following: the prediction mode, the quantization parameter (QP), the temporal layer, or the slice type. In some embodiments, whether to utilize the skip connection of multiple filters and / or the method of utilizing the skip connection of multiple filters depends on at least one of the following: the quantization step size, the block size of the video unit, the color component, or a signal in the bitstream. In some embodiments, the signal can be in at least one of the following: a sequence, a picture, a slice, a tile, a brick, a sub-picture, a codec tree unit (CTU), a codec tree block (CTB), a CTU row, a CTB row, a codec unit (CU), a codec block (CB), multiple CUs, or multiple CBs.

[0131] According to another embodiment of the present disclosure, a non-transitory computer-readable recording medium is provided. The non-transitory computer-readable recording medium stores a bitstream of a video generated by a method performed by a device for video processing. The method includes: applying a connection to a plurality of filters; applying a plurality of filter combinations with connections to a video unit; and generating a bitstream based on the filtered video unit.

[0132] According to some embodiments of the present disclosure, a method for storing a bitstream of a video is provided, the method comprising: applying a connection to a plurality of filters; applying a plurality of filter combinations with the connection to a video unit of the video; generating a bitstream based on the filtered video unit; and storing the bitstream in a non-transitory computer-readable recording medium.

[0133] Implementations of the present disclosure may be described according to the following clauses, features of which may be combined in any reasonable way.

[0134] Item 1. A method of video processing, comprising: applying a connection to a plurality of filters for conversion between a video unit of a video and a bitstream of the video; applying a combination of the plurality of filters having the connection to the video unit; and performing the conversion based on the filtered video unit.

[0135] Clause 2. The method of clause 1, wherein applying the connections to the plurality of filters comprises applying skip connections to the plurality of filters.

[0136] Item 3. A method according to Item 1, wherein the multiple filters include a first filter and a second filter, and wherein the input of the first filter and the output of the first filter are the input of the second filter, or wherein the input of the first filter and the output of the first filter are combined by the second filter.

[0137] Clause 4. The method of clause 3, wherein the first filter is one of: a neural network (NN) filter, a deblocking filter, or a sample adaptive offset (SAO) filter.

[0138] Clause 5. The method of clause 3, wherein the first filter is a combination of a NN filter and a deblocking filter.

[0139] Clause 6. The method of clause 3, wherein the second filter comprises at least one of: an adaptive loop filter (ALF), or a cross-component ALF (CCALF).

[0140] Clause 7. The method of Clause 1, wherein applying the connections to the plurality of filters comprises applying cross-component connections to the plurality of filters.

[0141] Clause 8. The method of clause 7, wherein filtering the first component is dependent on filtering the second component, or wherein a second filter filtering the second component is used to guide a first filter filtering the first component.

[0142] Clause 9. The method of clause 8, wherein the first component is a luma component and the second component is a chroma component, or wherein the first component is a chroma component and the second component is a luma component.

[0143] Clause 10. The method of clause 8, wherein the first component is a Y component and the second component is at least one of: a Cb component or a Cr component.

[0144] Clause 11. The method of clause 8, wherein the first component is a Cb component and the second component is at least one of: a Y component or a Cr component.

[0145] Clause 12. The method of clause 8, wherein the first component is a Cr component and the second component is at least one of: a Y component or a Cb component.

[0146] Clause 13. The method of clause 7, wherein the cross-component connection depends on at least one of: a codec sample of the video unit, a codec mode of the video unit, or codec statistics of the video unit.

[0147] Clause 14. The method of clause 13, wherein the codec statistics include at least one of: prediction mode, quantization parameter (QP), temporal layer, or slice type.

[0148] Clause 15. The method of clause 13, wherein samples before the second filter are used for the first filter, and / or wherein samples after the second filter are used for the first filter.

[0149] Clause 16. The method of clause 13, wherein information generated by the second filter is used for the first filter.

[0150] Clause 17. The method of clause 16, wherein the information is a boundary strength in a deblocking filter.

[0151] Clause 18. The method of Clause 13, wherein the first component is a luma component and the second component is a chroma component, or wherein the first component is a chroma component and the second component is a luma component.

[0152] Clause 19. The method of clause 13, wherein the first component is a Y component and the second component is at least one of the following: a Cb component or a Cr component, or wherein the first component is a Cb component and the second component is at least one of the following: a Y component or a Cr component.

[0153] Clause 20. The method of clause 7, wherein at least one filter of the plurality of filters is one of: a deblocking filter, a sample adaptive offset (SAO) filter, an ALF, or a NN filter.

[0154] Clause 21. The method of clause 1, wherein applying the connections to the plurality of filters comprises combining the plurality of filters via skip connections.

[0155] Clause 22. The method of clause 21, wherein a plurality of filters are involved in the skip connection.

[0156] Clause 23. The method of clause 22, wherein the number of filters involved in the skip connection is a positive integer or zero.

[0157] Clause 24. The method of clause 22, wherein the number of filters involved in the skip connection is 0, 1, 2, 3, 4, 5, 6 or 7.

[0158] Clause 25. The method of clause 21, wherein the jump connection is a short connection.

[0159] Clause 26. The method of clause 25, wherein a filter is involved in the skip connection.

[0160] Clause 27. The method of clause 21, wherein the jump connection is a long connection.

[0161] Clause 28. The method of clause 27, wherein filter sets with indices from i to j are involved in the skip connection, where i and j represent filter indices and are integers.

[0162] Clause 29. The method of clause 28, wherein i is less than or greater than j.

[0163] Clause 30. The method of clause 28, wherein the jump connection involves at least one of: a NN filter or a deblocking filter.

[0164] Clause 31. The method of clause 28, wherein samples prior to at least one of the NN filter or the deblocking filter are additional inputs to at least one of the ALF or the CCALF.

[0165] Clause 32. The method of clause 21, wherein the skip connections are applied to filter sets having indices from i to j, the filter sets being applied in serial order.

[0166] Clause 33. The method of clause 32, wherein an input of a first filter and an output of a second filter are inputs to a third filter, wherein the first filter is applied before or after the second filter.

[0167] Clause 34. The method of clause 33, wherein the first filter is a NN filter or a deblocking filter.

[0168] Clause 35. The method of clause 33, wherein the first filter is a combination of a set of filters.

[0169] Clause 36. The method of clause 35, wherein the first filter is a combination of a NN filter and a deblocking filter.

[0170] Clause 37. The method of clause 33, wherein the second filter is at least one of: a SAO filter or a cross-component SAO (CCSAO) filter.

[0171] Clause 38. The method of clause 33, wherein the third filter is at least one of: an ALF or a CCALF.

[0172] Clause 39. The method of clause 21, wherein the skip connection is applied to the first filter.

[0173] Clause 40. The method of clause 39, wherein the input of the first filter and the output of the second filter are inputs to a third filter.

[0174] Clause 41. The method of clause 40, wherein the first filter is one of: a NN filter, a deblocking filter, or a SAO filter.

[0175] Clause 42. The method of clause 40, wherein the first filter is a combination of a set of filters.

[0176] Clause 43. The method of clause 42, wherein the first filter is a combination of a NN filter and a deblocking filter.

[0177] Clause 44. The method of clause 40, wherein the third filter is at least one of an ALF or a CCALF.

[0178] Clause 45. The method of clause 21, wherein the skip connection is at least one of: an addition operation or a subtraction operation with a weighted parameter.

[0179] Clause 46. The method of clause 45, wherein O(x, y) = w 1 *f 1 (I i-1 (x,y))+w 2 *f 2 (I i (x,y)), and where I i (x,y) represents the output of the filter with index i, I i-1 (x,y) represents the input of the filter with index i, O(x,y) represents the output of the third filter, and w 1 and w 2 They represent weighting parameters, and f 1 and f 2Represent functions respectively.

[0180] Clause 47. The method according to clause 46, wherein f 1 (*) or f 2 At least one of (*) is a KxK filter with weighting parameters, where K is an integer.

[0181] Clause 48. The method according to clause 46, wherein w 1 or 2 At least one of them is equal to 1, or w 1 or 2 At least one item in is equal to 0.

[0182] Clause 49. The method according to clause 46, wherein w 1 and w 2 The sum is equal to a predefined value.

[0183] Clause 50. The method of clause 49, wherein the predefined value is equal to 1.

[0184] Clause 51. The method of clause 46, wherein samples around (x, y) are used in the function.

[0185] Clause 52. The method of clause 21, wherein the skip connection is a cascade operation.

[0186] Clause 53. The method of clause 21, wherein the skip connection is a fusion operation.

[0187] Clause 54. The method of clause 21, wherein the input of the third filter is I i-1 (x,y) and I i (x,y), and where I i (x,y) represents the output of the filter with index i, I i-1 (x,y) represents the input of the filter with index i.

[0188] Clause 55. The method of clause 21, wherein the input of the third filter is I j-1 (x,y) and I j (x,y), and where I j (x,y) represents the output of the filter with index j, I j-1 (x,y) represents the input of the filter with index j.

[0189] Clause 56. The method of clause 21, wherein the output of the third filter is the input I of the filter with index i. i-1 (x, y) and the output I of the filter with index ii The function f(*,*) of (x,y).

[0190] Clause 57. The method of clause 56, wherein f(*, *) is at least one of: an addition function or a subtraction function with weighting parameters.

[0191] Clause 58. The method of clause 56, wherein f(*,*) is a fusion function, or wherein f(*,*) is a cascade function.

[0192] Clause 59. The method according to clause 56, wherein O(x, y) = f(I i-1 (x,y),I i (x,y)).

[0193] Clause 60. The method of clause 56, wherein samples around (x, y) are used in the function.

[0194] Clause 61. The method of clause 60, wherein the sample is located at a predefined area, and the sample (x, y) is located at a center position of the predefined area.

[0195] Clause 62. The method of clause 61, wherein the predefined area is one of: a rhombus, a rectangle, or a quadrilateral.

[0196] Clause 63. The method of clause 62, wherein the predefined area has a size of K x K, where K is an integer.

[0197] Clause 64. The method according to clause 63, wherein K is one of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.

[0198] Clause 65. The method of clause 61, wherein the number of samples in the predefined area is L, where L is an integer.

[0199] Clause 66. The method according to clause 65, wherein L is one of 5, 13, 25, 41, 61.

[0200] Clause 67. The method of clause 60, wherein the parameters of the symmetric or diagonal samples are the same.

[0201] Clause 68. The method according to clause 60, wherein the sample I i-1 The shape of (x,y) and sample I j The shapes of (x,y) are different, or the sample I i-1 The shape of (x,y) and sample I j The shapes of (x,y) are the same, or the sample Ii-1 The number of (x,y) and sample I j The number of (x,y) is different, or the sample I i-1 The number of (x,y) and sample I j The number of (x,y) is the same.

[0202] Clause 69. The method of clause 56, wherein the filter with index i is one of: a NN filter, a deblocking filter, or a SAO filter.

[0203] Clause 70. The method of clause 56, wherein the filter with index i is a combination of a set of filters.

[0204] Clause 71. A method according to clause 70, wherein the filter with index i is a combination of a NN filter and a deblocking filter.

[0205] Clause 72. The method of clause 56, wherein the third filter is at least one of an ALF or a CCALF.

[0206] Clause 73. The method of clause 56, wherein the filter with index i uses a 5x5 or 3x3 filter.

[0207] Clause 74. The method of clause 56, wherein the third filter uses a 7x7 or 5x5 filter.

[0208] Clause 75. The method of clause 21, wherein the output of the third filter is the input I of the filter with index i. i-1 (x,y) and the output I of the filter with index j j The function f(*,*) of (x,y).

[0209] Clause 76. The method of clause 75, wherein f(*, *) is at least one of: an addition function or a subtraction function with weighting parameters.

[0210] Clause 77. A method according to Clause 75, wherein f(*,*) is a fusion function, or wherein f(*,*) is a cascade function.

[0211] Clause 78. The method of clause 75, wherein O(x, y) = f(I i-1 (x,y),I j (x,y)).

[0212] Clause 79. The method of clause 75, wherein samples around (x, y) are used in the function.

[0213] Clause 80. The method of clause 79, wherein the sample is located at a predefined area, and the sample (x, y) is located at a center position of the predefined area.

[0214] Clause 81. The method of clause 80, wherein the predefined area is one of: a rhombus, a rectangle, or a quadrilateral.

[0215] Clause 82. The method of clause 81, wherein the predefined area has a size of K x K, where K is an integer.

[0216] Clause 83. The method of clause 82, wherein K is one of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.

[0217] Clause 84. The method of clause 80, wherein the number of samples in the predefined area is L, where L is an integer.

[0218] Clause 85. The method according to clause 84, wherein L is one of 5, 13, 25, 41, 61.

[0219] Clause 86. The method of clause 79, wherein the parameters of the symmetric or diagonal samples are the same.

[0220] Clause 87. The method according to clause 79, wherein the sample I i-1 The shape of (x,y) and sample I j The shapes of (x, y) are different, or the sample I i-1 The shape of (x, y) is similar to that of the sample I j The shapes of (x, y) are the same, or the sample I i-1 The number of (x,y) and the sample I j The number of (x, y) is different, or the sample I i-1 The number of (x,y) and the sample I j The number of (x,y) is the same.

[0221] Clause 88. The method of clause 75, wherein the filter with index i is one of: a NN filter, a deblocking filter, or a SAO filter.

[0222] Clause 89. The method of clause 75, wherein the filter with index i is a combination of a set of filters.

[0223] Clause 90. The method of clause 89, wherein the filter with index i is a combination of a NN filter and a deblocking filter.

[0224] Clause 91. The method of clause 75, wherein the filter with index i is at least one of: a SAO filter or a CCSAO filter.

[0225] Clause 92. The method of clause 75, wherein the third filter is at least one of: an ALF or a CCALF.

[0226] Clause 93. The method of clause 21, wherein multiple skip connection operations are combined.

[0227] Clause 94. The method of clause 93, wherein the skip connection is applied to the filter with index i and a third filter, and another skip connection is applied to the filter with index j and the third filter.

[0228] Clause 95. A method according to Clause 1, wherein whether to utilize the jump connection of the multiple filters and / or the method of utilizing the jump connection of the multiple filters depends on at least one of the following: the codec sample of the video unit, the codec mode of the video unit, or the codec statistics of the video unit.

[0229] Clause 96. A method according to clause 95, wherein whether to utilize the skip connection of the multiple filters and / or the method of utilizing the skip connection of the multiple filters depends on at least one of the following: a prediction mode, a quantization parameter (QP), a temporal layer, or a slice type.

[0230] Clause 97. A method according to clause 95, wherein whether to utilize the jump connections of the multiple filters and / or the method of utilizing the jump connections of the multiple filters depends on at least one of the following: a quantization step size, a block size of the video unit, a color component, or a signal in the bitstream.

[0231] Clause 98. A method according to clause 97, wherein the signal is in at least one of the following: a sequence, a picture, a slice, a tile, a brick, a sub-picture, a codec tree unit (CTU), a codec tree block (CTB), a CTU row, a CTB row, a codec unit (CU), a codec block (CB), multiple CUs, or multiple CBs.

[0232] Clause 99. A method according to any one of clauses 1 to 98, wherein the converting comprises encoding the video unit into the bitstream.

[0233] Clause 100. A method according to any one of clauses 1 to 98, wherein the converting comprises decoding the video unit from the bitstream.

[0234] Clause 101. An apparatus for video processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform the method of any one of clauses 1 to 100.

[0235] Clause 102. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method of any of clauses 1 to 100.

[0236] Item 103. A non-volatile computer-readable storage medium storing a bitstream of a video generated by a method performed by a device for video processing, wherein the method comprises: applying a connection to a plurality of filters; applying the plurality of filter combinations having the connections to the video unit; and generating the bitstream based on the filtered video unit.

[0237] Item 104. A method for storing a bitstream of a video, comprising: applying a connection to a plurality of filters; applying the plurality of filter combinations having the connections to a video unit of the video; generating the bitstream based on the filtered video unit; and storing the bitstream in a non-transitory computer-readable recording medium. Example Device

[0238] Fig.19 A block diagram of a computing device 1900 in which various embodiments of the present disclosure may be implemented is shown. The computing device 1900 may be implemented as a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300), or may be included in a source device 110 (or video encoder 114 or 200) or a destination device 120 (or video decoder 124 or 300).

[0239] It should be understood that Fig.19 The computing device 1900 shown in the figure is for illustration purposes only and is not intended to in any way imply any limitation on the functionality and scope of the embodiments of the present disclosure.

[0240] like Fig.19 As shown, computing device 1900 includes a general computing device 1900. Computing device 1900 may include at least one or more processors or processing units 1910, memory 1920, storage unit 1930, one or more communication units 1940, one or more input devices 1950, and one or more output devices 1960.

[0241] In some embodiments, computing device 1900 can be implemented as any user terminal or server terminal with computing power. The server terminal can be a server provided by a service provider, a large computing device, etc. The user terminal can be, for example, any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet computer, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device, or any combination thereof, and includes accessories and peripherals of these devices, or any combination thereof. It is conceivable that computing device 1900 can support any type of interface to the user (such as a "wearable" circuit device, etc.).

[0242] The processing unit 1910 may be a physical processor or a virtual processor and may implement various processes based on a program stored in the memory 1920. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the computing device 1900. The processing unit 1910 may also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0243] The computing device 1900 typically includes various computer storage media. Such media can be any media accessible by the computing device 1900, including but not limited to volatile media and non-volatile media, or removable media and non-removable media. The memory 1920 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (such as a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) or flash memory) or any combination thereof. The storage unit 1930 can be any removable or non-removable medium, and can include machine-readable media, such as a memory, a flash drive, a disk, or other media that can be used to store information and / or data and can be accessed in the computing device 1900.

[0244] The computing device 1900 may also include additional removable / non-removable storage media, volatile / non-volatile storage media. Fig.19 Although not shown in the figure, a magnetic disk drive for reading from and / or writing to a removable nonvolatile magnetic disk and an optical disk drive for reading from and / or writing to a removable nonvolatile optical disk may be provided. In this case, each drive may be connected to the bus (not shown) via one or more data medium interfaces.

[0245] The communication unit 1940 communicates with another computing device via a communication medium. In addition, the functions of the components in the computing device 1900 can be implemented by a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device 1900 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or other general network nodes.

[0246] The input device 1950 may be one or more of various input devices, such as a mouse, keyboard, trackball, voice input device, etc. The output device 1960 may be one or more of various output devices, such as a display, a speaker, a printer, etc. With the help of the communication unit 1940, the computing device 1900 may also communicate with one or more external devices (not shown), such as storage devices and display devices, and the computing device 1900 may also communicate with one or more devices that enable a user to interact with the computing device 1900, or any device that enables the computing device 1900 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.), if necessary. Such communication may be performed via an input / output (I / O) interface (not shown).

[0247] In some embodiments, some or all components of the computing device 1900 may also be arranged in a cloud computing architecture rather than being integrated in a single device. In a cloud computing architecture, components may be provided remotely and work together to implement the functions described in the present disclosure. In some embodiments, cloud computing provides computing, software, data access and storage services, which will not require the end user to know the physical location or configuration of the system or hardware that provides these services. In various embodiments, cloud computing provides services via a wide area network (e.g., the Internet) using a suitable protocol. For example, a cloud computing provider provides applications via a wide area network, which can be accessed through a web browser or any other computing component. The software or components of the cloud computing architecture and the corresponding data may be stored on a remote server. The computing resources in a cloud computing environment may be merged or distributed at the location of a remote data center. Cloud computing infrastructure may provide services through a shared data center, although they appear as a single access point for users. Therefore, the cloud computing architecture may be used to provide the components and functions described herein from a service provider at a remote location. Alternatively, they may be provided by a conventional server, or installed directly or otherwise on a client device.

[0248] In an embodiment of the present disclosure, the computing device 1900 may be used to implement video encoding / decoding. The memory 1920 may include one or more video encoding / decoding modules 1925 having one or more program instructions. These modules can be accessed and executed by the processing unit 1910 to perform the functions of the various embodiments described herein.

[0249] In an example embodiment of performing video encoding, the input device 1950 may receive video data as input 1970 to be encoded. The video data may be processed by, for example, the video codec module 1925 to generate an encoded bitstream. The encoded bitstream may be provided as output 1980 via the output device 1960.

[0250] In an example embodiment of performing video decoding, the input device 1950 may receive an encoded bitstream as input 1970. The encoded bitstream may be processed by, for example, the video codec module 1925 to generate decoded video data. The decoded video data may be provided as output 1980 via the output device 1960.

[0251] Although the present disclosure has been specifically shown and described with reference to the preferred embodiments of the present disclosure, it will be understood by those skilled in the art that various changes can be made in form and detail without departing from the spirit and scope of the present application as defined by the appended claims. These changes are intended to be encompassed by the scope of the present application. Therefore, the foregoing description of the embodiments of the present application is not intended to be limiting.

Claims

1. A video processing method, comprising: Applying connections to a plurality of filters for conversion between a video unit of a video and a bitstream of the video; applying the plurality of filter combinations having the connections to the video unit; as well as The conversion is performed based on the filtered video unit.

2. The method of claim 1 , wherein applying the concatenation to the plurality of filters comprises: A skip connection is applied to the plurality of filters.

3. The method of claim 1 , wherein the plurality of filters comprises a first filter and a second filter, and wherein the input of the first filter and the output of the first filter are the input of the second filter, or Wherein an input of the first filter and an output of the first filter are combined by the second filter.

4. The method of claim 3, wherein the first filter is one of: Neural Network (NN) Filters, Deblocking filter, or Sample Adaptive Offset (SAO) filter. The method of claim 3 , wherein the first filter is a combination of a NN filter and a deblocking filter.

6. The method of claim 3, wherein the second filter comprises at least one of the following: Adaptive loop filter (ALF), or Cross-Component ALF (CCALF).

7. The method of claim 1 , wherein applying the concatenation to the plurality of filters comprises: Cross-component connections are applied to the plurality of filters.

8. A method according to claim 7, wherein filtering the first component is dependent on filtering the second component, or The second filter for filtering the second component is used to guide the first filter for filtering the first component.

9. The method of claim 8, wherein the first component is a luminance component and the second component is a chrominance component, or The first component is a chrominance component and the second component is a luminance component.

10. The method of claim 8, wherein the first component is a Y component, and the second component is at least one of the following: a Cb component or a Cr component.

11. The method of claim 8, wherein the first component is a Cb component, and the second component is at least one of the following: a Y component or a Cr component.

12. The method of claim 8, wherein the first component is a Cr component, and the second component is at least one of the following: a Y component or a Cb component.

13. The method of claim 7, wherein the cross-component connection depends on at least one of the following: Codec samples of the video unit, the codec mode of the video unit, or The encoding and decoding statistics of the video unit.

14. The method according to claim 13, wherein the encoding and decoding statistics include at least one of the following: Prediction model, Quantization parameter (QP), Time layer, or Strip type.

15. The method of claim 13, wherein samples before the second filter are used for the first filter, and / or Wherein the samples after the second filter are used for the first filter. The method of claim 13 , wherein information generated by the second filter is used in the first filter. The method of claim 16 , wherein the information is a boundary strength in a deblocking filter.

18. The method of claim 13, wherein the first component is a luma component and the second component is a chroma component, or The first component is a chrominance component and the second component is a luminance component.

19. The method of claim 13, wherein the first component is a Y component, and the second component is at least one of the following: a Cb component or a Cr component, or The first component is a Cb component, and the second component is at least one of the following: a Y component or a Cr component.

20. The method of claim 7, wherein at least one filter of the plurality of filters is one of: Deblocking filter, Sample Adaptive Offset (SAO) filter, ALF, or NN filter.

21. The method of claim 1 , wherein applying the concatenation to the plurality of filters comprises: The plurality of filters are combined via skip connections.

22. The method of claim 21, wherein a plurality of filters are involved in the skip connection.

23. The method according to claim 22, wherein the number of filters involved in the skip connection is a positive integer or zero.

24. The method of claim 22, wherein the number of filters involved in the skip connection is 0, 1, 2, 3, 4, 5, 6 or 7.

25. The method of claim 21, wherein the skip connection is a short connection.

26. The method of claim 25, wherein a filter is involved in the skip connection.

27. The method of claim 21, wherein the skip connection is a long connection.

28. The method according to claim 27, wherein a filter set having an index from i to j is involved in the skip connection, where i and j represent filter indices and are integers.

29. The method of claim 28, wherein i is less than or greater than j.

30. The method according to claim 28, wherein the skip connection involves at least one of the following: a NN filter or a deblocking filter.

31. The method of claim 28, wherein samples prior to at least one of the NN filter or the deblocking filter are additional inputs to at least one of the ALF or the CCALF.

32. The method of claim 21, wherein the skip connections are applied to filter sets having indices from i to j, the filter sets being applied in serial order.

33. The method of claim 32, wherein the input of the first filter and the output of the second filter are inputs to the third filter, Wherein the first filter is applied before or after the second filter.

34. The method of claim 33, wherein the first filter is a NN filter or a deblocking filter.

35. The method of claim 33, wherein the first filter is a combination of a set of filters.

36. The method of claim 35, wherein the first filter is a combination of a NN filter and a deblocking filter.

37. The method of claim 33, wherein the second filter is at least one of: a SAO filter or a cross-component SAO (CCSAO) filter.

38. The method of claim 33, wherein the third filter is at least one of: ALF or CCALF.

39. The method of claim 21, wherein the skip connection is applied to a first filter.

40. The method of claim 39, wherein the input of the first filter and the output of the second filter are inputs to a third filter.

41. The method of claim 40, wherein the first filter is one of: a NN filter, a deblocking filter, or a SAO filter.

42. The method of claim 40, wherein the first filter is a combination of a set of filters.

43. The method of claim 42, wherein the first filter is a combination of a NN filter and a deblocking filter.

44. The method of claim 40, wherein the third filter is at least one of an ALF or a CCALF.

45. The method of claim 21, wherein the skip connection is at least one of: an addition operation or a subtraction operation with a weighted parameter.

46. ​​The method of claim 45, wherein O(x,y)=w1*f1(I i-1 (x,y))+w2*f2(I i (x,y)), and where I i (x,y) represents the output of the filter with index i, I i-1 (x, y) represents the input of the filter with index i, O(x, y) represents the output of the third filter, w1 and w2 represent weighting parameters, respectively, and f1 and f2 represent functions, respectively.

47. The method of claim 46, wherein at least one of f1(*) or f2(*) is a KxK filter with weighting parameters, where K is an integer.

48. The method of claim 46, wherein at least one of w1 or w2 is equal to 1, or wherein at least one of w1 or w2 is equal to 0.

49. The method of claim 46, wherein the sum of w1 and w2 is equal to a predefined value.

50. The method of claim 49, wherein the predefined value is equal to 1.

51. The method of claim 46, wherein samples around (x, y) are used in the function.

52. The method of claim 21, wherein the skip connection is a cascade operation.

53. The method of claim 21, wherein the skip connection is a fusion operation.

54. The method of claim 21, wherein the input to the third filter is I i-1 (x,y) and I i (x,y), and where I i (x,y) represents the output of the filter with index i, I i-1 (x,y) represents the input of the filter with index i.

55. The method of claim 21, wherein the input to the third filter is I j-1 (x,y) and I j (x,y), and where I j (x,y) represents the output of the filter with index j, I j-1 (x,y) represents the input of the filter with index j.

56. The method of claim 21, wherein the output of the third filter is the input I of the filter with index i i-1 (x, y) and the output I of the filter with index i i The function f(*,*) of (x,y).

57. The method of claim 56, wherein f(*, *) is at least one of: an addition function or a subtraction function with weighting parameters.

58. The method of claim 56, wherein f(*,*) is a fusion function, or Where f(*,*) is the concatenation function.

59. The method of claim 56, wherein O(x, y) = f(I i-1 (x,y),I i (x,y)).

60. The method of claim 56, wherein samples around (x, y) are used in the function.

61. The method of claim 60, wherein the sample is located at a predefined area, and the sample (x, y) is located at a center position of the predefined area.

62. The method of claim 61, wherein the predefined area is one of: a diamond, a rectangle, or a quadrilateral.

63. The method of claim 62, wherein the size of the predefined area is K x K, where K is an integer.

64. The method of claim 63, wherein K is one of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.

65. The method of claim 61, wherein the number of samples in the predefined area is L, where L is an integer.

66. The method of claim 65, wherein L is one of 5, 13, 25, 41, 61.

67. The method of claim 60, wherein parameters of symmetric or diagonal samples are the same.

68. The method of claim 60, wherein sample I i-1 The shape of (x,y) and sample I j The shapes of (x,y) are different, or Among them, sample I i-1 The shape of (x,y) and sample I j (x,y) have the same shape, or Among them, sample I i-1 The number of (x,y) and sample I j The number of (x,y) is different, or Among them, sample I i-1 The number of (x,y) and sample I j The number of (x,y) is the same.

69. The method of claim 56, wherein the filter with index i is one of: a NN filter, a deblocking filter, or a SAO filter.

70. The method of claim 56, wherein the filter with index i is a combination of a set of filters.

71. The method of claim 70, wherein the filter with index i is a combination of a NN filter and a deblocking filter.

72. The method of claim 56, wherein the third filter is at least one of an ALF or a CCALF.

73. The method of claim 56, wherein the filter with index i uses a 5x5 or 3x3 filter.

74. The method of claim 56, wherein the third filter uses a 7x7 or 5x5 filter.

75. The method of claim 21, wherein the output of the third filter is the input I of the filter with index i i-1 (x,y) and the output I of the filter with index j j The function f(*,*) of (x,y).

76. The method of claim 75, wherein f(*, *) is at least one of: an addition function or a subtraction function with weighting parameters.

77. The method of claim 75, wherein f(*,*) is a fusion function, or Where f(*,*) is the concatenation function.

78. The method of claim 75, wherein O(x, y) = f(I i-1 (x,y),I j (x,y)).

79. The method of claim 75, wherein samples around (x, y) are used in the function.

80. The method of claim 79, wherein the sample is located at a predefined area, and the sample (x, y) is located at a center position of the predefined area.

81. The method of claim 80, wherein the predefined area is one of: a diamond, a rectangle, or a quadrilateral.

82. The method of claim 81, wherein the size of the predefined area is K x K, where K is an integer.

83. The method of claim 82, wherein K is one of 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14.

84. The method of claim 80, wherein the number of samples in the predefined area is L, where L is an integer.

85. The method of claim 84, wherein L is one of 5, 13, 25, 41, 61.

86. The method of claim 79, wherein parameters of symmetric or diagonal samples are the same.

87. The method of claim 79, wherein sample I i-1 The shape of (x,y) and sample I j The shapes of (x,y) are different, or The sample I i-1 The shape of (x, y) is similar to that of the sample I j (x,y) have the same shape, or The sample I i-1 The number of (x,y) and the sample I j The number of (x,y) is different, or The sample I i-1 The number of (x,y) and the sample I j The number of (x,y) is the same.

88. The method of claim 75, wherein the filter with index i is one of: a NN filter, a deblocking filter, or a SAO filter.

89. The method of claim 75, wherein the filter with index i is a combination of a set of filters.

90. The method of claim 89, wherein the filter with index i is a combination of a NN filter and a deblocking filter.

91. The method of claim 75, wherein the filter with index i is at least one of: a SAO filter or a CCSAO filter.

92. The method of claim 75, wherein the third filter is at least one of: ALF or CCALF.

93. The method of claim 21, wherein multiple skip connection operations are combined.

94. The method of claim 93, wherein the skip connection is applied to a filter with index i and a third filter, and another skip connection is applied to a filter with index j and the third filter.

95. The method of claim 1, wherein whether to utilize the skip connections of the plurality of filters and / or the method of utilizing the skip connections of the plurality of filters depends on at least one of the following: Codec samples of the video unit, the codec mode of the video unit, or The encoding and decoding statistics of the video unit.

96. The method of claim 95, wherein whether to utilize skip connections of the plurality of filters and / or the method of utilizing skip connections of the plurality of filters depends on at least one of: Prediction model, Quantization parameter (QP), Time layer, or Strip type.

97. The method of claim 95, wherein whether to utilize skip connections of the plurality of filters and / or the method of utilizing skip connections of the plurality of filters depends on at least one of: Quantization step size, the block size of the video unit, color components, or signal in the bitstream.

98. The method of claim 97, wherein the signal is at least one of: sequence, picture, Strips, Tile, Bricks, Sub-picture, Codec Tree Unit (CTU), Codec Tree Block (CTB), CTU line, CTB line, Codec Unit (CU), Codec Block (CB), Multiple CUs, or Multiple CBs.

99. The method of any one of claims 1 to 98, wherein the converting comprises encoding the video unit into the bitstream.

100. The method of any one of claims 1 to 98, wherein the converting comprises decoding the video unit from the bitstream.

101. An apparatus for video processing, comprising a processor and a non-volatile memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to perform a method according to any one of claims 1 to 100.

102. A non-transitory computer-readable storage medium storing instructions that cause a processor to perform the method according to any one of claims 1 to 100.

103. A non-transitory computer-readable storage medium storing a bitstream of a video generated by a method performed by an apparatus for video processing, wherein the method comprises: Apply concatenation to multiple filters; applying the plurality of filter combinations having the connections to the video unit; as well as The bitstream is generated based on the filtered video unit.

104. A method for storing a bitstream of a video, comprising: Apply concatenation to multiple filters; applying the plurality of filter combinations having the connections to a video unit of the video; generating the bitstream based on the filtered video unit; as well as The bit stream is stored in a non-transitory computer-readable recording medium.

Images