Video encoding / decoding loop filtering method, video decoding method and device

Abstract

An embodiment of the present disclosure provides a video encoding / decoding loop filtering method, a video decoding method and an apparatus, the method including: filtering a reconstructed image of a current image using a first filter to obtain a first reconstructed video unit; filtering the reconstructed image using a second filter to obtain a second reconstructed video unit; determining filter coefficients based on neighborhood information contained in convolution templates of the first reconstructed video unit and the second reconstructed video unit; and filtering the first reconstructed video unit, the second reconstructed video unit and the filter coefficients based on the convolution template and the filter coefficients to obtain a third reconstructed video unit. This solves the problem of low filtering adaptation effect due to the weighted fusion operation for a single pixel point in the related art. The convolution filtering method adds more neighboring pixel information as much as possible in the filtering correction process to make the reconstructed value closer to the original value, thereby achieving good output image quality.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This disclosure is based on and claims priority to Chinese patent application CN202211182900.3, filed on September 27, 2022, entitled "Video encoding and decoding loop filtering method, video decoding method and apparatus," the entire disclosure of which is incorporated herein by reference.

[0002] [Technical field] TECHNICAL FIELD The embodiments of the present disclosure relate to the field of communications, and in particular to a video encoding / decoding loop filtering method, a video decoding method and an apparatus. [Background technology]

[0003] Neural network in-loop filtering (NNF) can be added as a new module to loop filtering, or can be used as a replacement for the original deblocking filter (DBF) and sample adaptive offset (SAO) modules, or can be used in conjunction with conventional filtering modules. Generally, after NNF filtering, adaptive loop filtering (ALF) is still used in encoders and decoders to further improve image performance.

[0004] NNF filtering generally uses offline training, and the weights obtained through offline training are applied to VTM (the VVC reference software testing platform) for effective use. Traditional networks are generally offline networks, and do not refer to the original image when actually used. This means that NNF filters can be either too strong or too weak. Therefore, NNF filtering must be adapted based on the characteristics of the current image.

[0005] One of the adaptive methods is to introduce a scaling factor to perform a strength adjustment (Scale) operation. The Scale operation uses a least-squares straight line fitting method to approximate the reconstructed frame after NNF filtering to the original frame, achieving an adaptive effect between the NNF filtering strength and the original image. Furthermore, a slice-level or coding block-level switch operation can be added to control the effective range of NNF filtering.

[0006] Another adaptation method is to add DBF filtering and perform weighting, and the weighting solution includes using only the output of DBF, using NN filtering solution and DBF filtering weighting, etc., and the pixel point involved in the weighting is the current pixel point to be adapted, and this solution can be flexibly selected and controlled at the slice level or coding block level.

[0007] However, since the above two adaptive methods are both weighted fusion operations for a single pixel point, the filtering adaptation effect is low, and there is still room for further improvement in the correction of NNF filtering. At present, no effective solution to this problem has been proposed. Summary of the Invention [Problem to be solved by the invention]

[0008] The embodiments of the present disclosure provide a video encoding / decoding loop filtering method, a video decoding method and an apparatus to solve the problem of low filtering adaptation effect due to weighted fusion operations on at least a single pixel point in the related art. [Means for solving the problem]

[0009] According to one embodiment of the present disclosure, filtering the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; filtering the reconstructed image with a second filter to obtain a second reconstructed video unit; determining filter coefficients based on neighborhood information included in convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is information about neighboring pixel points centered on a pixel point in the convolution template; filtering the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain a third reconstructed video unit.

[0010] According to another embodiment of the present disclosure, obtaining a bitstream including an encoded video sequence and obtaining loop filtering control information corresponding to a loop filtering scheme, the loop filtering control information including filter coefficients; filtering the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; filtering the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; The present invention further provides a video decoding method, which includes a step of filtering the first reconstructed video unit and the second reconstructed video unit based on the convolution templates of the first reconstructed video unit and the filter coefficients of the second reconstructed video unit to obtain a third reconstructed video unit.

[0011] According to another embodiment of the present disclosure, a first filtering module configured to filter the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; a second filtering module configured to filter the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; a first determination module configured to determine filter coefficients based on neighborhood information included in convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is information about neighboring pixel points centered on a pixel point in the convolution template; and a third filtering module configured to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain a third reconstructed video unit.

[0012] According to another embodiment of the present disclosure, an acquisition module configured to acquire a bitstream including an encoded video sequence to acquire loop filtering control information corresponding to a loop filtering scheme; a first filtering module configured to filter the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; a second filtering module configured to filter the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; The video decoding device further includes a third filtering module configured to filter the first reconstructed video unit and the second reconstructed video unit based on convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain a third reconstructed video unit.

[0013] According to yet another embodiment of the present disclosure, there is further provided a computer-readable storage medium having stored thereon a computer program, the computer program being configured, when executed, to perform the steps of any of the method embodiments described above.

[0014] According to another embodiment of the present disclosure, there is further provided an electronic device including a memory having a computer program stored therein, and a processor configured to execute the computer program to perform the steps of any of the method embodiments described above. [Effects of the Invention]

[0015] In an embodiment of the present disclosure, a reconstructed image of a current image is filtered using a first filter to obtain a first reconstructed video unit, and the reconstructed image is filtered using a second filter to obtain a second reconstructed video unit. Filter coefficients are determined based on neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, where the neighborhood information is information about neighboring pixel points centered on a pixel point in the convolution template. The first reconstructed video unit and the second reconstructed video unit are filtered based on the convolution template and the filter coefficients to obtain a third reconstructed video unit. In this way, the problem of low filtering adaptation effect due to the weighted fusion operation for a single pixel point in the related art can be solved. The convolution filtering method adds as much neighboring pixel information as possible in the filtering correction process to make the reconstructed value closer to the original value, thereby achieving good output image quality. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 10 is a hardware configuration block diagram of a mobile terminal for a video encoding / decoding loop filtering method according to an embodiment of the present disclosure. [Figure 2] 1 is a flowchart of a video encoding / decoding loop filtering method according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram (part 1) of a convolution model of CCA filtering according to the present embodiment. [Figure 4] FIG. 1 is a schematic diagram (part 1) of adaptive filtering by convolution on the encoding side according to the present embodiment. [Figure 5] FIG. 10 is a schematic diagram (part 2) of adaptive filtering by convolution on the encoding side according to the present embodiment. [Figure 6] FIG. 10 is a schematic diagram (part 3) of adaptive filtering by convolution on the encoding side according to the present embodiment. [Figure 7]FIG. 10 is a schematic diagram (part 4) of adaptive filtering by convolution on the encoding side according to the present embodiment. [Figure 8] FIG. 5 is a schematic diagram (part 5) of adaptive filtering by convolution on the encoding side according to the present embodiment. [Figure 9] FIG. 10 is a schematic diagram (part 6) of adaptive filtering by convolution on the encoding side according to the present embodiment. [Figure 10] FIG. 2 is a schematic diagram (part 2) of a convolution model of CCA filtering according to the present embodiment. [Figure 11] FIG. 10 is a schematic diagram (part 3) of a convolution model of CCA filtering according to the present embodiment. [Figure 12] FIG. 4 is a schematic diagram (part 4) of a convolution model of CCA filtering according to the present embodiment. [Figure 13] 1 is a flowchart of a video decoding method according to an embodiment of the present disclosure. [Figure 14] FIG. 10 is a schematic diagram of a filtering adaptation operation by convolution on the decoding side according to the present embodiment. [Figure 15] 1 is a block diagram of a video encoding / decoding loop filtering device according to an embodiment of the present disclosure. [Figure 16] FIG. 1 is a block diagram of a video decoding device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. It should be noted that terms such as "first" and "second" in the specification, claims and drawings of this disclosure are not used to describe a particular order or chronology, but are used to distinguish between similar objects.

[0018] An embodiment of a method according to the present disclosure may be implemented in a mobile terminal, a computer terminal, or a similar computing device. Taking an example of operation on a mobile terminal, FIG. 1 is a block diagram of a hardware configuration of a mobile terminal for implementing a video encoding / decoding loop filtering method according to an embodiment of the present disclosure. As shown in FIG. 1, the mobile terminal includes one or more processors 102 (only one of which is shown in FIG. 1 ) (the processor 102 may include, but is not limited to, a processing unit such as a microprocessor MCU or a programmable logic device) and memory 104 for storing data. The mobile terminal may also include a transmission device 106 and an input / output device 108 for performing communication functions. As will be appreciated by those skilled in the art, the structure shown in FIG. 1 is merely a schematic diagram and does not limit the structure of the mobile terminal. For example, the mobile terminal may include more or fewer components than those shown in FIG. 1 or may have a different configuration than that shown in FIG. 1.

[0019] The memory 104 may store software programs and modules of application software, such as a computer program corresponding to a video encoding / decoding loop filtering method according to an embodiment of the present disclosure. The processor 102 executes the computer programs stored in the memory 104 to perform various functional applications and video encoding / decoding loop filtering, i.e., to implement the method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, a corporate intranet, a local area network, a mobile communication network, and combinations thereof.

[0020] The transmission device 106 transmits and receives data via a network. An example of the network may include a wireless network provided by a carrier of the mobile terminal. In one embodiment, the transmission device 106 includes a network adapter (abbreviated as a Network Interface Controller, NIC) that is connected to other network devices via a base station and can communicate with the Internet. In one embodiment, the transmission device 106 may be a radio frequency (abbreviated as RF) module that communicates with the Internet wirelessly.

[0021] In this embodiment, a video encoding / decoding loop filtering method is provided, which is executed on the above-mentioned mobile terminal or network architecture. FIG. 2 is a flowchart of the video encoding / decoding loop filtering method according to an embodiment of the present disclosure. As shown in FIG. 2, the video encoding / decoding loop filtering method is applied to the encoding / decoding side. The flow includes the following steps S202 to S208:

[0022] In step S202, the reconstructed image of the current image is filtered by a first filter to obtain a first reconstructed video unit.

[0023] In this embodiment, the above step S202 may specifically include a step of performing a deblocking filter DBF on the reconstructed image, or a step of performing a sample adaptive offset SAO on the reconstructed image, or a step of performing a deblocking filter DBF and a sample adaptive offset SAO on the reconstructed image, or a step of performing a luminance mapping LMCS process based on chroma scaling on the reconstructed image.

[0024] In step S204, the reconstructed image is filtered by a second filter to obtain a second reconstructed video unit.

[0025] In this embodiment, the above step S204 may specifically include the following steps: performing neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; performing neural network loop filtering on the reconstructed image and performing a weighted addition of the neural network loop filtering result and the first reconstructed video unit to obtain the second reconstructed video unit; performing neural network loop filtering on the reconstructed image and performing intensity adjustment on the neural network loop filtering result based on the original image to obtain the second reconstructed video unit; or performing other loop filtering other than the neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of a deblocking filter DBF, a sample adaptive offset SAO, and a luminance mapping LMCS based on chroma scaling.

[0026] In step S206, filter coefficients are determined based on neighborhood information contained in the convolution templates of the first reconstructed video unit and the second reconstructed video unit, where the neighborhood information is information about adjacent pixel points centered on a pixel point in the convolution template.

[0027] In step S208, the first reconstructed video unit and the second reconstructed video unit are filtered according to the convolution template and the filter coefficients to obtain a third reconstructed video unit.

[0028] The above steps S202 to S208 can solve the problem of low filtering adaptation effect due to the weighted fusion operation for a single pixel point in the related art. The convolution filtering method adds as much neighboring pixel information as possible in the filtering correction process, so that the reconstructed value can be more closely approximated to the original value, and achieve good output image quality.

[0029] In one embodiment, step S208 may specifically include the steps of: for each pixel point in the video unit of the reconstructed image and the video unit of the original image, respectively determining a difference value between each pixel point and a pixel point at each neighboring position in the convolution template; multiplying the difference values ​​respectively to obtain a first pixel product; determining a difference value between the pixel point in the video unit of the original image and the pixel point in the video unit of the reconstructed image to obtain a covariance; and determining the filter coefficient based on the covariance based on a Wiener filtering method.

[0030] In another embodiment, the above step S208 may specifically further include the steps of: calculating a difference value between the central pixel point and a neighboring pixel point of the convolution template based on the first reconstructed video unit, and multiplying the difference value by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a first result; and simultaneously calculating a difference value between the central pixel point and a neighboring pixel point of the convolution template based on the second reconstructed video unit, and multiplying the difference value by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a second result; and adding the first result, the second result, and the second reconstructed video unit, and performing rounding to obtain the third reconstructed video unit.

[0031] The above S202 may specifically be DBF filtering, the above S204 may specifically be NN filtering, and the above S208 may specifically be CCA filtering. Accordingly, the third reconstructed video unit is a CCAF filtering result, specifically, the third reconstructed video unit can be obtained by performing the filtering process in the following manner, and the third reconstructed video unit is the following CCAF filtering result: NNout'=int((NNout C +w0*(DBFout t -NNout C )+w1*(DBFout l -NNout C )+w2*(DBFout c -NNout C )+w3*(DBFout r -NNout C )+w4*(DBFout b -NNout C )+w5*(NNout t -NNout C )+w6*(NNout l -NNout C )+w7*(NNout r -NNout C )+w8*(NNout b -NNout C ))>>BITS_NUM+0.5), NNout' is the CCAF filtering result, DBFout is the first reconstructed video unit, NNout is the second reconstructed video unit, and w i is the i-th filter coefficient, w0 to w4 are filter coefficients for the first reconstructed video unit, and w5 to w8 are filter coefficients for the second reconstructed video unit.

[0032] In this embodiment, the video unit of the reconstructed image includes a reconstructed image, or a slice, or a coding block, or a coding tree block.

[0033] In another embodiment, the above step S208 may specifically further include at least one of the following steps: performing a filtering process on the luminance components of the first reconstructed video unit and the second reconstructed video unit based on the filter coefficients based on the convolution template; performing a filtering process on the first chroma components of the first reconstructed video unit and the second reconstructed video unit based on the filter coefficients based on the convolution template; and performing a filtering process on the second chroma components of the first reconstructed video unit and the second reconstructed video unit based on the filter coefficients based on the convolution template.

[0034] In a preferred embodiment, the method further includes a step of determining a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image; a step of determining a second loss between pixels of a video unit of the reconstructed image before filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of the original image; and a step of determining whether to perform CCA filtering on the video unit of the current image based on the first loss and the second loss.

[0035] In another preferred embodiment, the method further includes a step of determining a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of the original image; a step of filtering with predetermined filter coefficients and determining a third loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of the original image; and a step of determining, based on the first loss and the third loss, to perform the filtering process using the filter coefficients or to perform the filtering process using predetermined filter coefficients.

[0036] Specifically, a first loss after filtering is performed using the filter coefficient is determined, a second loss before filtering is performed using the filter coefficient is determined, or a third loss after filtering is performed using a predetermined filter coefficient is determined, and whether to perform CCA filtering is determined based on the first loss and the second loss, or the first loss and the third loss. If the first loss is smaller than the second loss, or if the first loss is smaller than the third loss, it is determined to perform CCA filtering and a CCA filtering identifier is set to 1. If the first loss is equal to or greater than the second loss, or if the first loss is equal to or greater than the third loss, it is determined not to perform CCA filtering and a CCA filtering identifier is set to 0.

[0037] Furthermore, if the filter coefficients are filter coefficients corresponding to each slice of the reconstructed image, when the first loss is smaller than the second loss, it is determined to perform CCA filtering for each slice of the reconstructed image based on the filter coefficients corresponding to the slice, and when the first loss is equal to or greater than the second loss, it is determined not to perform CCA filtering for each slice of the reconstructed image based on the filter coefficients corresponding to the slice; or when the first loss is smaller than the second loss, it is determined to perform CCA filtering for each block of the reconstructed image based on the filter coefficients corresponding to the slice to which it belongs, and when the first loss is equal to or greater than the second loss, it is determined not to perform CCA filtering for each block of the reconstructed image based on the filter coefficients corresponding to the slice to which it belongs, wherein the blocks are one of one or more CTUs, one or more CUs, and pixel blocks of a predetermined size.

[0038] In another preferred embodiment, the method further includes the steps of determining a first loss between pixels of the video unit of the reconstructed image after filtering and pixels of the original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; determining a second loss between pixels of the video unit of the reconstructed image before filtering and pixels of the original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; performing the filtering on the video unit of the current image based on filter coefficients of neighboring video units of the video unit of the current image to determine a fourth loss between pixels of the video unit of the reconstructed image after filtering and pixels of the original image; and determining filter coefficients to be used for the video unit of the current image based on the first loss, the second loss, and the fourth loss, and performing the filtering based on the determined filter coefficients.

[0039] In another preferred embodiment, the method further includes the steps of determining a first loss between pixels of the video unit of the reconstructed image after filtering and pixels of the original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; determining a second loss between pixels of the video unit of the reconstructed image before filtering and pixels of the original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; determining whether to perform the filtering on the video unit of the current image based on the first loss and the second loss, recalculating filter coefficients for the filtered video unit, and again determining whether to perform filtering based on the first loss and the second loss, and after repeating N times, confirming whether to perform the filtering on the video unit of the current image.

[0040] The convolution template in this embodiment includes one of a diamond template, a square template, and a cross template, specifically, a 3*3 diamond template, a 5*5 diamond template, a 7*7 diamond template, a 3*3 square template, a 5*5 square template, a 7*7 square template, and a 5*5 cross template, and some commonly used templates are listed here, but are not limited to the scope of protection of this embodiment.

[0041] In one embodiment, the method further includes a step of encoding the filter coefficients and the reconstructed image to obtain a video code stream, wherein a pps_cca_enabled_flag of the video code stream is an identifier of the CCA filtering, the pps_cca_enabled_flag being 1 or 0, where 1 indicates that CCA filtering is performed on the current slice and 0 indicates that CCA filtering is not performed on the current slice; or, assuming that the pps_cca_enabled_flag is 1, a cca_block_flag of the video code stream identifies whether CCA filtering is performed on the current block, the cca_block_flag being 1 or 0, where 1 indicates that CCA filtering is performed on the current block and 0 indicates that CCA filtering is not performed on the current block.

[0042] In this embodiment, pps_cca_info_in_ph_flag of the video codec stream indicates the location of the filter coefficient, and pps_cca_info_in_ph_flag is 1 or 0, where 1 indicates that the filter coefficient is in the picture header and 0 indicates that the filter coefficient is in the slice header.

[0043] In another embodiment, step S206 may specifically include performing a CCA filtering process on a slice or block whose CCA filtering identifier is 1, based on the first filtering result, the second filtering result, and a filter coefficient corresponding to each slice, to obtain the CCA filtering result. Specifically, obtaining a first sub-filtering result of a luminance component and / or a chrominance component in the first filtering result, obtaining a second sub-filtering result of a luminance component and / or a chrominance component in the second filtering result, and performing a CCA filtering process on each slice or each block of the reconstructed image based on the first sub-filtering result, the second sub-filtering result, and the filter coefficient, to obtain the CCA filtering result.

[0044] In one embodiment, the method further includes encoding the filter coefficients and the reconstructed image to obtain a video code stream, wherein a cca_luma_enabled_flag of the video code stream identifies whether to perform CCA filtering on a luminance component of a current slice, the cca_luma_enabled_flag being 1 or 0, where 1 indicates that CCA filtering is performed on the luminance component of the current slice and 0 indicates that CCA filtering is not performed on the luminance component of the current slice; and a cca_cb_enabled_flag identifies whether to perform CCA filtering on a chroma Cb component of the current slice, the cca_cb_enabled_flag being 1 or 0, where 1 indicates that CCA filtering is performed on the chroma Cb component of the current slice and 0 indicates that CCA filtering is not performed on the chroma Cb component of the current slice. cca_cr_enabled_flag identifies whether to perform CCA filtering on the chroma Cr component of the current slice, where cca_cr_enabled_flag is 1 or 0, where 1 indicates that CCA filtering is performed on the chroma Cr component of the current slice, and 0 indicates that CCA filtering is not performed on the chroma Cr component of the current slice; or cca_block_flag[i] identifies whether to perform CCA filtering on the luma or chroma component of the current block, where cca_block_flag[i] is 1 or 0, where 1 indicates that CCA filtering is performed on the luma or chroma component of the current block, and 0 indicates that CCA filtering is not performed on the luma or chroma component of the current block, where i is 0 indicates the luma component, i is 1 indicates the chroma Cb component, and i is 2 indicates the chroma Cr component.

[0045] In another embodiment, the method further includes a step of: for a target block whose CCA filtering identifier is 1, determining target filter coefficients of the target block; determining a first image cost after CCA filtering processing and a second image cost before CCA filtering processing of each block in the target block; and setting the CCA filtering identifier of the target block whose first image cost is smaller than the second image cost to 1; and performing the steps N times to obtain the CCA filtering identifier and corresponding filter coefficients of each block.

[0046] In another embodiment, the method further includes a step of performing the following steps for each block of the reconstructed image in a coefficient fusion mode to update the filter coefficients of each block, if the filter coefficients are filter coefficients corresponding to each block of the reconstructed image, where the block being performed is called the current block: The steps include determining a first image cost for pixel points of the current block after performing CCA filtering using the filter coefficients; if a filter coefficient exists in the block adjacent to the top of the current block, determining a fourth image cost for the current block to be the image cost after performing CCA filtering using the filter coefficients on the block adjacent to the top of the current block; if a filter coefficient exists in the block adjacent to the left of the current block, determining a fifth image cost for the current block to be the first image cost after performing CCA filtering using the filter coefficients on the block adjacent to the left of the current block; determining a sixth image cost for the current block without performing CCA filtering using the filter coefficients on the current block; and selecting a filter coefficient corresponding to the smallest image cost from the first, fourth, fifth, and sixth image costs as the filter coefficient for the current block, where the block is one of one or more CTUs, one or more CUs, or a pixel block of a predetermined size.

[0047] In another embodiment, the method includes encoding the filter coefficients and the reconstructed image to obtain a video code stream, wherein, assuming that the CCA filtering identifier is 1, a cca_block_flag of the video code stream identifies whether to perform CCA filtering on a current block, the cca_block_flag being 1 or 0, where 1 indicates to perform CCA filtering on the current block and 0 indicates not to perform CCA filtering on the current block; if the cca_block_flag is 1, a cca_block_merge_flag[i][Blk AddrX][Blk AddrY] identifies whether the current block uses the coefficient merging mode, the cca_block_merge_flag[i][Blk AddrX][Blk AddrY] being 1 or 0, where 1 indicates to use the coefficient merging mode; if the cca_block_merge_flag[i][Blk AddrX][Blk AddrY] is 1, a cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] The method further includes the step of: cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] identifies the type of coefficient merging mode of the current block, and cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] is 1 or 0, where 1 indicates using the filter coefficients of the adjacent block above, and 0 indicates using the filter coefficients of the adjacent block to the left, and where i is 0 indicates the luma component, i is 1 indicates the chroma Cb component, and i is 2 indicates the chroma Cr component.

[0048] In this embodiment, a convolutional filtering adaptive algorithm (CCAF) is implemented by introducing multi-point neighboring pixels into the NN filtering correction process. The existing solution is a weighted fusion operation for a single pixel point. Therefore, the CCAF convolutional filtering method adds as much neighboring pixel information as possible to the NN filtering correction process, making the reconstructed value closer to the original value and achieving good output image quality.

[0049] In this embodiment, a coding-side convolutional filtering adaptation algorithm is implemented, and the flow of the coding-side CCA filtering method includes the following steps 1 to 3.

[0050] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in recBuf.

[0051] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0052] In step 3, the convolution filtering adaptive algorithm is executed. The results stored in the two bufs in the above steps are sent to the CCAF stage, which then combines them with the original image to calculate the CCAF filtering coefficients and perform the filtering operation.

[0053] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, CCAF filtering coefficients are calculated based on the covariance, and a filtering operation is performed based on the convolution template.

[0054] The neighborhood information in this embodiment is neighborhood information centered on the position pixel point in the CCF convolution template.

[0055] The neighborhood information participating in the calculation of the CCA filtering coefficients includes neighborhood information of the position pixel point in the Rec image, neighborhood information of the position pixel point in the NNin image, and neighborhood information of the position pixel point in the NNout image.

[0056] As a preferred embodiment, Fig. 3 is a schematic diagram (part 1) of a convolution model of CCA filtering according to this embodiment, and as shown in Fig. 3, the neighborhood information of CCA filtering is as follows: c represents the pixel to be filtered at the middle position, t represents the upper neighboring pixel, b represents the lower neighboring pixel, l represents the left neighboring pixel, and r represents the right neighboring pixel.

[0057] Figure 4 is a schematic diagram (part 1) of the encoding-side convolutional filtering adaptation in this embodiment. As shown in Figure 4, the encoding side applies the convolutional filtering adaptation algorithm as shown in the figure above. The output (DBFout) of a conventional filtering module (e.g., DBF) and the output (NNout) of a NN filtering module are input to a CCA module, and a multi-point weighting operation is performed on the output of the conventional module and the output of the NN module.

[0058] The present invention is not limited to DBF filtering and NN filtering, but can be extended to any two filtering methods of adaptive algorithms.

[0059] Step 1 is DBF filtering, that is, DBF filtering is performed on the reconstructed image Rec after LMCS, and the result of the DBF filtering is stored in dbfoutBuf.

[0060] Step 2 is NN filtering, that is, after obtaining the reconstructed image of LMCS, NN filtering is performed.

[0061] First, various data that need to be input to the network, such as reconstructed image samples, QP information, CU partition information, deblocking filtering information, and predicted samples, are set, and then this information is input into the NN filtering network module to obtain reconstructed image samples after NN filtering processing. The NN filtering results are stored in NNoutBuf.

[0062] In step 3, the convolutional filtering adaptive algorithm is executed, and the results stored in the two Bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation. Specifically, it includes the following steps 3-a to 3-c.

[0063] The principle of the method using Wiener filtering is as follows. The method for calculating the mean square error (MSE) after filtering is as follows:

[0064]

number

[0065] n indicates the number of coefficients, c is the coefficient to be found, and p is the offset of the center relative coordinate. The above formula is decomposed.

[0066]

number

[0067]

number

[0068]

number

[0069] Based on the above equations, the coefficients can be solved to obtain the optimal performance for one pixel point.

[0070] For multiple pixel points in one slice, the following equation can be formulated:

[0071]

number

[0072] This equation is solved to obtain the filtering coefficients. Step 3-a calculates the covariance and includes the following (1) to (2).

[0073] (1) For each pixel point, calculate the difference between each position of the convolution template and the center point, and multiply the difference values. At the same time, calculate the org value of the current point - the pixel value of the current point.

[0074] (2) For each pixel point in one slice, perform the calculation in (1). In the main solution, the filtering template is set as the template shown in FIG. 3 (the template is not limited to this and can be expanded or contracted).

[0075] In step 3-b, the CCA filtering coefficients are calculated based on the covariance. Based on the covariance calculated in step 3-a, equations are formulated and solved to obtain the filtering coefficients (c0~c8).

[0076]

number

[0077] Since the coefficients obtained directly by the formula are floating-point numbers, the coefficients are converted into integers (w0 to w8).

[0078]

number

[0079] Here, BITS_NUM is 7. o(i) is the pixel value of the reconstructed image, r(i) is the pixel value of the image after NN filtering, pi is the position offset between a pixel point and an adjacent pixel point, and shifting a coefficient to the left converts the coefficient to an integer.

[0080] In step 3-c, after constructing the CCA output signal and obtaining the calculated filtering coefficients, the filtering operation is performed according to the following formula:

[0081]

number

[0082] DBFout refers to the output by DBF, NNout refers to the output by NN filter, and w i indicates the i-th filtering coefficient, w0 to w4 are filter coefficients output to the DBF, and w5 to w8 are filter coefficients output to the NN.

[0083] (In step 3-b, the coefficients are shifted to the left, so they must be shifted back to the right after use.) Degenerate to DBF and NN weighted solutions.

[0084] The neighbor information is deleted, and all other w coefficients except for w2 are 0.

[0085]

number

[0086] Degenerate to scale: remove neighbor information and set all w coefficients other than w2 to 0. DBFout c Rec c (NNin, which is also the previous reconstruction value of DBF).

[0087]

number

[0088] In preferred embodiment 1, the CCA input is modified. Currently, there are multiple combinations of NN filtering modules and traditional filtering modules, so the input of the CCA module is not necessarily limited to the method in the main solution. Output 1 and Input 2 can be changed separately.

[0089] In method 1-1, CCA input 1 is modified, specifically, Rec is modified. In the Rec main solution, Rec indicates the output of DBF. In actual operation, it may be the reconstructed image after conventional filtering modules such as DBF and SAO, or the output before DBF.

[0090] Rec indicates a reconstructed image before DBF, and FIG. 5 is a schematic diagram (part 2) of adaptive filtering by convolution on the encoding side according to this embodiment, which includes the following steps 1 to 3 as shown in FIG.

[0091] In step 1, the reconstructed image after LMCS is stored in recBuf. In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0092] Step 3 executes the convolution filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0093] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0094] Rec indicates the reconstructed image after DBF and SAO, and FIG. 6 is a schematic diagram (part 3) of the coding-side convolution filtering adaptation according to this embodiment. As shown in FIG. 6, the steps are as follows:

[0095] In step 1, DBF and SAO filtering are performed on the reconstructed image after LMCS, and the result after SAO filtering is stored in saooutBuf.

[0096] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0097] Step 3 executes the convolution filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0098] For the pixel points of one slice, the filtering coefficient calculation is performed based on the convolution template, the CCA filtering coefficient is calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0099] In Method 1-2, CCA input 2 may be changed and NN-related may be changed. In the main solution, NN-related indicates the output of the NN. However, since the NN is offline, a weighting operation with the reconstructed pixels is generally performed after the NN. Therefore, NN-related may be the output of the NN.

[0100] NNrelated indicates the output of the NN and the weighting result of the DBF, and FIG. 7 is a schematic diagram (part 4) of the filtering adaptation by convolution on the encoding side according to this embodiment, which includes the following steps 1 to 3 as shown in FIG.

[0101] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0102] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0103] The pixel values ​​in recBuf and the pixel values ​​in nnoutBuf are weighted, and the weighted result is stored in nnoutBuf.

[0104]

number

[0105] The range of k is [0,1], and may be, for example, 0.75, 0.5, etc. Step 3 executes the convolution filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0106] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0107] NNrelated indicates the result after the NN has scaled, and FIG. 8 is a schematic diagram (part 5) of the filtering adaptation by convolution on the encoding side according to this embodiment, which includes the following steps 1 to 3 as shown in FIG.

[0108] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in recBuf.

[0109] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0110] A scale operation is performed on the pixel values ​​after LMCS (i.e., the input of the NN) and the pixel values ​​in nnoutBuf, and the result is stored in nnoutBuf. The scale may be calculated using the least squares method or may be a predetermined value.

[0111]

number

[0112] Step 3 executes the convolution filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0113] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0114] In methods 1-3, a broader scope of CCA may be used, and CCA is not limited to being used in a neural network-based coding-decoding framework, but may also be used in a conventional coding-decoding framework.

[0115] For example, a CCA operation can be performed using a reconstructed image that has not undergone DBF and a reconstructed image that has undergone DBF as two inputs to the CCA. Figure 9 is a schematic diagram (part 6) of the filtering adaptation by convolution on the encoding side according to this embodiment, and as shown in Figure 9, it includes the following steps 1 to 3.

[0116] In step 1, the reconstructed image after LMCS is stored in recBuf. In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in dbfoutBuf.

[0117] Step 3 executes the CCA adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0118] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0119] In the second preferred embodiment, a CCA decision mechanism is implemented. Since the coefficients need to be calculated at the encoding side and then transmitted to the decoding side, the transmission overhead of the video encoding code stream may increase. If there are too many pixels, calculating one set of coefficients for one slice may result in no performance gain or performance degradation for some pixel points. Therefore, the present solution provides various decision mechanism means.

[0120] In the preferred embodiment 2-1, switch control is performed. Slice level switch: Perform DBF filtering on the reconstructed image after LMCS, and store the result after DBF filtering in dbfoutBuf. Perform NN filtering on the reconstructed image after LMCS, and store the result of NN filtering in nnoutBuf. Execute a convolution-based filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage, where coefficients are calculated and filtering operations are performed.

[0121] For pixel points in one slice, the covariance is calculated based on the neighborhood information included in the convolution template, and the CCA filtering coefficient is calculated based on the covariance. The slice-level switch control includes the following (1) to (3).

[0122] (1) Calculate the cost (cost indicates RD cost) of the adaptation coefficient obtained in 3-b.

[0123] (2) Calculate the cost of the reconstructed image without using CCA. (3) For the current slice, the cost with CCA and the cost without CCA are competed to identify and transmit the optimal solution. If there is no gain or degradation in performance after CCA, the identifier is set to 0, indicating that the current slice does not perform the CCA operation; otherwise, the identifier is set to 1, indicating that the current slice performs the CCA operation.

[0124] For slices with a flag of 1, a CCA operation is performed according to the method described in step 2-b of Example 1 to obtain a reconstructed image. For slices with a flag of 0, no processing is performed.

[0125] Block-level switch: Perform DBF filtering on the reconstructed image after LMCS, and store the result after DBF filtering in dbfoutBuf. Perform NN filtering on the reconstructed image after LMCS, and store the result of NN filtering in nnoutBuf. Execute a convolution-based filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage, where coefficients are calculated and filtering operations are performed.

[0126] For pixel points of one slice, the covariance is calculated based on the neighborhood information included in the convolution template, and the CCA filtering coefficient is calculated based on the covariance. The block-level switch control includes the following (1) to (3).

[0127] (1) Each block calculates the cost (cost indicates RD cost) using the adaptive coefficient obtained in 3-b.

[0128] (2) Calculate the cost of the reconstructed image without CCA for each block. (3) For each block, the cost with CCA and the cost without CCA are compared to identify and transmit the optimal solution. If there is no gain or degradation in performance after CCA, the identifier is set to 0, indicating that this block does not perform CCA operation; otherwise, the identifier is set to 1, indicating that this block performs CCA operation.

[0129] For blocks with a flag of 1, a CCA operation is performed according to the method described in step 2-b of Example 1 to obtain a reconstructed image. For blocks with a flag of 0, no processing is performed.

[0130] The coefficient optimization solution in this embodiment involves performing DBF filtering on the reconstructed image after LMCS and storing the DBF filtering result in dbfoutBuf. Then, performing NN filtering on the reconstructed image after LMCS and storing the NN filtering result in nnoutBuf. A convolution-based filtering adaptive algorithm is implemented. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0131] For pixel points of one slice, the covariance is calculated based on the neighborhood information included in the convolution template, and the CCA filtering coefficient is calculated based on the covariance, and the coefficient is optimized, which includes the following steps (1) to (2).

[0132] (1) Calculate the cost before and after using CCA for each block (a block may refer to a CTU or an operation block of a NN). If there is no gain or degradation in performance after CCA, set the identifier to 0 to indicate that this block does not perform CCA operation; otherwise, set the identifier to 1 to indicate that this block performs CCA operation.

[0133] (2) For the block whose identifier is 1, recalculate a set of coefficients, and then repeat step (1). According to this step, the coefficients are iterated N times (here, N is not limited and can be 3), and each time, the cost corresponding to this coefficient is calculated, and compared to obtain the optimized coefficients and the final identifier of each block.

[0134] For blocks with a flag of 1, a CCA operation is performed according to the method described in step 2-b of Example 1 to obtain a reconstructed image. For blocks with a flag of 0, no processing is performed.

[0135] In the preferred embodiment 2-2, competitive determination is performed. Competitive screening of the predetermined coefficients and the adaptive coefficients includes the following steps 1 to 3.

[0136] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0137] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0138] Step 3 executes the convolution filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0139] For pixel points of one slice, calculate the covariance based on the neighborhood information contained in the convolution template, and calculate the CCA filtering coefficient based on the covariance. The competitive decision includes the following steps (1) to (3):

[0140] (1) Calculate the cost (cost indicates RD cost) of the adaptation coefficient obtained in 3-b.

[0141] (2) In 3-c, the cost is calculated for a predetermined coefficient (the predetermined coefficient is set based on statistical information).

[0142] (3) Competing the adaptive coefficients with the predetermined coefficients, the optimal solution is identified and transmitted.

[0143] Filtering is performed according to step 3-c of Example 1 using coefficients with small costs.

[0144] In a third preferred embodiment, block-level CCA is performed. In the above solutions, all coefficients are at the slice level, and using only one set of coefficients for each pixel of one slice may not be highly adaptable. Therefore, this solution proposes a block-level CCA technique. The block-level CCA operation steps include the following steps 1 to 3:

[0145] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0146] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0147] In step 3, the convolutional filtering adaptation algorithm is executed. The results stored in the two bufs in the above steps are sent to the CCA stage in block units to calculate the coefficients and perform the filtering operation.

[0148] For each pixel point of a block, calculate the covariance based on the neighborhood information contained in the convolution template, calculate the CCA filtering coefficient based on the covariance, and perform the filtering operation for the current block based on the convolution template. The specific operation is the same as above, so the description is omitted here.

[0149] Add block-level switch control. In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0150] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0151] In step 3, the convolutional filtering adaptation algorithm is executed. The results stored in the two bufs in the above steps are sent to the CCA stage in block units to calculate the coefficients and perform the filtering operation.

[0152] For pixel points in one block, calculate the covariance based on the neighborhood information contained in the convolution template, calculate the CCA filtering coefficients based on the covariance, and compare the cost before and after using CCA for each block. If there is no gain or degradation in performance after CCA, set the identifier to 0 to indicate that this block does not perform a CCA operation; otherwise, set the identifier to 1 to indicate that this block performs a CCA operation.

[0153] For blocks with a flag of 1, a CCA operation is performed according to the method described in step 3-c of Example 1 to obtain a reconstructed image. For blocks with a flag of 0, no processing is performed.

[0154] Fusion mode: Because the overhead of block-level transmission coefficients is large, a coefficient fusion mode is added to save some transmission overhead. For the current block, since the neighborhood has a certain correlation, the current block can use the coefficients calculated by the neighboring blocks above or to the left to complete the CCA filtering of the current block. The steps are as follows:

[0155] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0156] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0157] In step 3, the convolutional filtering adaptation algorithm is executed. The results stored in the two bufs in the above steps are sent to the CCA stage in block units to calculate the coefficients and perform the filtering operation.

[0158] For pixel points in one block, the covariance is calculated based on the neighborhood information contained in the convolution template, and the CCA filtering coefficients are calculated based on the covariance. The main steps of the fusion include the following steps (1) to (5):

[0159] (1) Calculate the cost using the accommodation coefficient obtained in 3-b. (2) If there are CCA coefficients in the block above the current block, calculate the cost of directly using the coefficients of the block above the current block.

[0160] (3) If there are CCA coefficients in the block to the left of the current block, calculate the cost of directly using the coefficients of the block to the left of the current block.

[0161] (4) Calculate the cost of not performing CCA. (5) Compare the four costs above and select the optimal solution for the current block.

[0162] For blocks with a flag of 1, a CCA operation is performed according to the method described in step 3-c of Example 1 to obtain a reconstructed image. For blocks with a flag of 0, no processing is performed.

[0163] In a fourth preferred embodiment, the convolution template is modified. The main solution is a 3x3 diamond template. Expanding the module allows for more information to be integrated in the CCA process, while shrinking the template can save bits in the CCA transmission process. Based on this, the template has many more selection solutions.

[0164] The following solution only modifies the shape of the convolution template, and the overall operation steps are as follows:

[0165] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0166] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0167] Step 3 executes the convolution filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0168] For the pixel points of one block, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0169] In this embodiment, the template is not limited to a 3x3 diamond template. Figure 10 is a schematic diagram (part 2) of the convolution model of CCA filtering in this embodiment. As shown in Figure 10, the template may be a 5x5 template, a 7x7 template, etc.

[0170] In this embodiment, the template is not limited to a diamond shape, and may be a square or other shape, as shown in FIG. 11, which is a schematic diagram (part 3) of a convolution model of CCA filtering according to this embodiment.

[0171] FIG. 12 is a schematic diagram (part 4) of the convolution model of CCA filtering according to this embodiment, and as shown in FIG. 12, the shapes of Rec and the NN template may be different.

[0172] In this embodiment, the center point of the main solution is the center point of the NN template, which may be the center point of Rec.

[0173]

number

[0174] In the fifth preferred embodiment, different component CCA and inter-component CCA are performed, and the CCA method does not limit the components, and luminance and chrominance can be used or not used respectively.

[0175] Both luma and chroma use CCA, and the steps are as follows: In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0176] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0177] Step 3 executes the convolution filtering adaptation algorithm. The results of the luminance and chrominance components stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0178] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0179] For luminance only, CCA is used and the steps are as follows: In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0180] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0181] Step 3 executes the convolution filtering adaptation algorithm. The luminance component results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0182] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0183] For chroma only, use CCA and the steps are as follows: In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0184] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0185] Step 3 executes the convolution filtering adaptation algorithm. The chroma component results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0186] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, the CCA filtering coefficients are calculated based on the covariance, and the filtering operation is performed based on the convolution template.

[0187] For the cross-component CCA mode, generally, considering that luma contains more information than chroma, luma can be used to calculate the CCA filtering coefficients, which includes the following steps 1 to 3:

[0188] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0189] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0190] Step 3 executes the convolution filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage to calculate the coefficients and perform the filtering operation.

[0191] For pixel points in one slice, the covariance is calculated based on the neighborhood information contained in the convolution template, and the CCA filtering coefficients are calculated based on the covariance to construct a CCA output signal.

[0192] After obtaining the calculated filtering coefficients, the filtering operation is performed according to the following formula:

[0193]

number

[0194] DBFout_luma refers to the output of the DBF for the luminance component, NNout_luma refers to the output of the NN filter for the luminance component, and NNout_chroma C refers to the output of the NN filter for the chroma component, and w i indicates the i-th filtering coefficient, w0 to w4 are filter coefficients output to the DBF, and w5 to w8 are filter coefficients output to the NN.

[0195] According to another embodiment of the present disclosure, a video decoding method is further provided, and FIG. 13 is a flowchart of the video decoding method according to the embodiment of the present disclosure. As shown in FIG. 13, the flow includes the following steps S1302 to S1308.

[0196] In step S1302, a bitstream including an encoded video sequence is obtained to obtain loop filtering control information corresponding to a loop filtering scheme, where the loop filtering control information includes a filter coefficient.

[0197] In step S1304, the reconstructed image of the current image is filtered by a first filter to obtain a first reconstructed video unit.

[0198] In this embodiment, the above step S1304 may specifically include: performing a deblocking filter DBF on the reconstructed image; performing a sample adaptive offset SAO on the reconstructed image; performing a deblocking filter DBF and a sample adaptive offset SAO on the reconstructed image; or performing a luminance mapping LMCS process based on chroma scaling on the reconstructed image, which is similar to the above step S202.

[0199] In step S1306, the reconstructed image of the current image is filtered by a second filter to obtain a second reconstructed video unit.

[0200] In this embodiment, step S1306 may specifically include: performing neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; performing neural network loop filtering on the reconstructed image and weighting and adding a result of the neural network loop filtering and the first reconstructed video unit to obtain the second reconstructed video unit; performing neural network loop filtering on the reconstructed image and performing intensity adjustment on the result of the neural network loop filtering based on the original image to obtain the second reconstructed video unit; or performing other loop filtering other than neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit, where the other loop filtering includes at least one of a deblocking filter DBF, a sample adaptive offset SAO, and a luminance mapping based on chroma scaling LMCS. This is similar to step S204.

[0201] In step S1308, the first reconstructed video unit and the second reconstructed video unit are filtered according to the convolution templates and filter coefficients of the first reconstructed video unit and the second reconstructed video unit to obtain a third reconstructed video unit.

[0202] The above steps S1302 to S1308 can solve the problem of low filtering effect caused by weighted fusion operation for a single pixel point in the related art. The convolution filtering method adds as much neighboring pixel information as possible in the filtering correction process, so that the reconstructed value can be more closely approximated to the original value, and achieve good output image quality.

[0203] In this embodiment, the above step S1308 may further include the steps of: calculating a difference between the central pixel point and neighboring pixel points of the convolution template based on the first reconstructed video unit, and multiplying the difference between the central pixel point and neighboring pixel points by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a first result; simultaneously calculating a difference between the central pixel point and neighboring pixel points of the convolution template based on the second reconstructed video unit, and multiplying the difference between the central pixel point and neighboring pixel points by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a second result; and adding the first result, the second result, and the second reconstructed video unit and performing rounding to obtain the third reconstructed video unit, which is similar to the above step S208.

[0204] The above S1304 may specifically be DBF filtering, the above S1306 may specifically be NN filtering, and the above S1308 may specifically be CCA filtering. Accordingly, the third reconstructed video unit is a CCAF filtering result, specifically, the third reconstructed video unit can be obtained by performing a filtering process in the following manner, and the third reconstructed video unit is the following CCAF filtering result:

[0205]

number

[0206] where NNout' is the CCAF filtering result, DBFout is the first reconstructed video unit, NNout is the second reconstructed video unit, and w i is the i-th filter coefficient, w0 to w4 are filter coefficients for the first reconstructed video unit, and w5 to w8 are filter coefficients for the second reconstructed video unit.

[0207] In this embodiment, the loop filtering information includes at least one of: a sequence level filtering switch identifier indicating whether to turn on / off the loop filtering method for the video sequence to which the current video frame belongs; a filtering image level identifier indicating whether the loop filtering model information appears in image header information or slice header information; and loop filtering model structure information indicating whether to perform the loop filtering for the video unit to which the loop filtering method is applied.

[0208] In one embodiment, the method further includes determining a value of the sequence level filtering switch identifier; and determining to turn on the loop filtering for the current encoded video sequence if the value of the sequence level filtering switch identifier is a first value, and determining to turn off the loop filtering for the current encoded video sequence if the value of the sequence level filtering switch identifier is a second value, wherein the first value is different from the second value.

[0209] In one embodiment, the method further includes the steps of determining a value of the filtering picture level identifier, and determining that the loop filtering model information appears in a picture header if the value of the filtering picture level identifier is a third value, and determining that the loop filtering model information appears in a slice header if the value of the filtering picture level identifier is a fourth value, wherein the third value is different from the fourth value.

[0210] In this embodiment, the video unit is , image or slice Includes: In this embodiment, the loop filtering model structure information includes at least one of: a video unit filtering switch identifier that instructs a video unit to turn the loop filtering scheme on / off; a video unit component switch identifier that instructs a video unit component to turn the loop filtering scheme on / off; a video unit block level filtering switch identifier that instructs each level in the video unit to turn the loop filtering scheme on / off; a video unit filtering coefficient absolute value that indicates a filtering coefficient absolute value corresponding to each level in the video unit; and a video unit filtering coefficient symbol that indicates a filtering coefficient symbol corresponding to each level in the video unit, and the video unit component includes at least one of a luminance component of a video unit, a first chroma component of a video unit, and a second chroma component of a video unit.

[0211] In one embodiment, the method further includes determining a value of the video unit filtering switch identifier and, based on the value, determining whether to turn on the loop filtering or turn off the loop filtering for the current video unit.

[0212] In one embodiment, the method further includes determining a value of the video unit luma component switch identifier and determining, based on the value, whether to turn on the loop filtering for a current video unit luma component or turn off the loop filtering for a current video unit luma component; determining a value of a first chroma component switch identifier of the video unit and determining, based on the value, whether to turn on the loop filtering for a first chroma component of the current video unit or turn off the loop filtering for the first chroma component of the current video unit; and determining a value of a second chroma component switch identifier of the video unit and determining, based on the value, whether to turn on the loop filtering for a second chroma component of the current video unit or turn off the loop filtering for the second chroma component of the current video unit.

[0213] In one embodiment, the method further includes determining a value of the video unit block level filtering switch identifier and, based on the value, determining whether to turn on the loop filtering for coding blocks in the video unit or to turn off the loop filtering for coding blocks in the video unit.

[0214] In one embodiment, the method further includes filtering the video unit with the loop filtering scheme turned on based on the video unit filtering coefficient absolute value and the video unit filtering coefficient symbol.

[0215] In this embodiment, the loop filtering model structure information further includes a video unit block level filter coefficient fusion mode switch identifier indicating whether to turn on a coefficient fusion mode for a coding block and a component of the coding block in a video unit, and a video unit block level filter coefficient fusion mode type identifier indicating the position of an adjacent coding block of a current coding block in a video unit, where the current coding block performs the loop filtering using the filter coefficients of the adjacent coding block, and the coding block components include at least one of a luminance component, a first chroma component, and a second chroma component.

[0216] In this embodiment, the loop filtering model structure information further includes a cross-component filtering mode switch identifier indicating whether to turn on a cross-component filtering mode for a coding block and a component of the coding block in a video unit.

[0217] On the other hand, in the above step S1306, the CCA filtering identifier can be obtained from the pps_cca_enabled_flag of the video codec stream, and whether to perform CCA filtering is determined based on the CCA filtering identifier, where the pps_cca_enabled_flag is 1 or 0, where 1 indicates that CCA filtering is performed for the current slice, and 0 indicates that CCA filtering is not performed for the current slice. If the pps_cca_enabled_flag is 1, the filter coefficient is obtained from the video codec stream. Alternatively, if the pps_cca_enabled_flag is 1, determine whether to perform CCA filtering on each block of the reconstructed image based on the cca_block_flag of the video code stream, where the cca_block_flag is 1 or 0, where 1 indicates that CCA filtering is performed on the current block and 0 indicates that CCA filtering is not performed on the current block, and if the cca_block_flag is 1, obtain the filter coefficients for each block, where the block is one or more CTUs, one or more CUs, or a pixel block of a predetermined size.

[0218] On the other hand, in the above step S1306, specifically, the location of the filter coefficient may be determined by the pps_cca_info_in_ph_flag of the video code stream, where pps_cca_info_in_ph_flag is 1 or 0, where 1 indicates that the filter coefficient is in the picture header and 0 indicates that the filter coefficient is in the slice header; if pps_cca_info_in_ph_flag is 1, the filter coefficient is obtained from the picture header; and if pps_cca_info_in_ph_flag is 0, the filter coefficient is obtained from the slice header.

[0219] In another embodiment, the above step S1308 may specifically include the steps of: obtaining a first sub-filtering result of the luminance component and / or the chrominance component in the first filtering result for a slice or block whose CCA filtering identifier is 1; and obtaining a second sub-filtering result of the luminance component and / or the chrominance component in the second filtering result; and performing a CCA filtering process on each slice or each block of the reconstructed image based on the first sub-filtering result, the second sub-filtering result, and the filter coefficients to obtain the CCA filtering result.

[0220] In another embodiment, step S1308 specifically includes the steps of: determining whether to perform CCA filtering on the luminance component of the current slice based on a cca_luma_enabled_flag of the video codec stream; performing CCA filtering on the luminance component of the current slice if the cca_luma_enabled_flag is 1; and not performing CCA filtering on the luminance component of the current slice if the cca_luma_enabled_flag is 0; determining whether to perform CCA filtering on the chroma Cb components of the current slice based on a cca_cb_enabled_flag; performing CCA filtering on the chroma Cb components of the current slice if the cca_cb_enabled_flag is 1; and not performing CCA filtering on the chroma Cb components of the current slice if the cca_cb_enabled_flag is 0; and or determining whether to perform CCA filtering on the luma or chroma components of the current block based on cca_block_flag[i], and if cca_block_flag[i] is 1, performing CCA filtering on the luma or chroma components of the current block, and if cca_block_flag[i] is 0, not performing CCA filtering on the luma or chroma components of the current block, wherein i is 0 when it indicates the luma component, i is 1 when it indicates the chroma Cb component, and i is 2 when it indicates the chroma Cr component.

[0221] In another embodiment, the step S1308 specifically includes the steps of: if the CCA filtering identifier is 1, determining whether to perform CCA filtering on the current block according to cca_block_flag of the video code stream; if the cca_block_flag is 1, performing CCA filtering on the current block; if the cca_block_flag is 0, not performing CCA filtering on the current block; if the cca_block_flag is 1, determining whether the current block uses the coefficient merging mode according to cca_block_merge_flag[i][Blk AddrX][Blk AddrY]; if the cca_block_merge_flag[i][Blk AddrX][Blk AddrY] is 1, determining to use the coefficient merging mode; if the cca_block_merge_flag[i][Blk AddrX][Blk AddrY] is 0, determining not to use the coefficient merging mode; If cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] is 1, determining a type of coefficient fusion mode for the current block based on cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY]; if cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] is 1, performing CCA filtering on the current block using the filter coefficients of the adjacent block above; if cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] is 0, performing CCA filtering on the current block using the filter coefficients of the adjacent block to the left, wherein i indicates a luma component when i is 0, a chroma Cb component when i is 1, and a chroma Cr component when i is 2.

[0222] In this embodiment, the convolution template may be a 3*3 diamond template, a 5*5 diamond template, a 7*7 diamond template, a 3*3 square template, a 5*5 square template, a 7*7 square template, or a 5*5 cross template. Here, some commonly used templates are listed, but are not limited to the scope of protection of this embodiment.

[0223] In this embodiment, a convolutional filtering adaptive algorithm is implemented on the decoding side, and the overall flow of the CCA filtering method on the decoding side is as follows:

[0224] In step 1, DBF filtering is performed on the reconstructed image after LMCS, and the result of DBF filtering is stored in dbfoutBuf.

[0225] In step 2, NN filtering is performed on the reconstructed image after LMCS, and the result of the NN filtering is stored in nnoutBuf.

[0226] Step 3 executes the convolutional filtering adaptation algorithm. The results stored in the two bufs in the above steps are sent to the CCA stage, which analyzes the CCA coefficients from the codestream and performs the filtering operation.

[0227] In 3-a, the decoder reads the filter coefficients based on syntax elements in the codestream.

[0228] In step 3-b, the output after CCA filtering is reconstructed, and the operation method is the same as step 3-c in Example 1.

[0229] 14 is a schematic diagram of a convolution-based filtering adaptation operation on the decoding side according to this embodiment. As shown in FIG. 14, a convolution-based filtering adaptation algorithm is executed on the decoding side, and the contents of the filter coefficients and the like are obtained by analyzing the code stream. The decoding-side CCAF includes the following steps 1 to 3.

[0230] Step 1 is DBF filtering, that is, DBF filtering is performed on the reconstructed image Rec after LMCS, and the result of the DBF filtering is stored in dbfoutBuf.

[0231] Step 2 is NN filtering, that is, after obtaining the reconstructed image of LMCS, NN filtering is performed.

[0232] First, various data that need to be input to the network, such as reconstructed image samples, QP information, CU partition information, deblocking filtering information, predicted samples, etc., are set, and then this information is input to the network module to obtain reconstructed image samples after NN processing. The results of NN filtering are stored in nnoutBuf.

[0233] In step 3, a convolutional filtering adaptive algorithm is executed, and the results stored in the two bufs in the above steps are sent to the CCA stage, where the CCA coefficients are analyzed from the codestream for filtering. The specific steps include the following steps 3-a to 3-b:

[0234] In step 3-a, filtering coefficients are calculated and w0 to w8 are analyzed from the codestream.

[0235] In step 3-b, the CCA output signal is constructed and the filtering coefficients are obtained, and then the filtering operation is performed according to the following equation:

[0236]

number

[0237] DBFout refers to the output by DBF, NNout refers to the output by NN filter, and w iindicates the i-th filtering coefficient, w0 to w4 are filter coefficients output to the DBF, and w5 to w8 are filter coefficients output to the NN.

[0238] In this embodiment, the syntax elements related to CCA in the video coding codestream transmitted by the convolutional filtering adaptation algorithm include:

[0239] SPS layer: The encoder determines whether the current sequence uses a CCA solution based on the sps_cca_enabled_flag syntax element in the SPS, with syntax semantics shown in Table 1 below.

[0240] [Table 1]

[0241] When sps_cca_enabled_flag is 1, it indicates that the current sequence uses the CCA technique, and when it is 0, it indicates that the current sequence does not use the CCA technique.

[0242] PPS layer: The encoder determines the location of the CCA solution related information in the syntax elements based on the pps_cca_info_in_ph_flag syntax element in the PPS, and the syntax semantics are shown in Table 2 below.

[0243] [Table 2]

[0244] If pps_cca_info_in_ph_flag is 1, it indicates that the CCA-related information appears in the picture header, and if it is 0, it indicates that the CCA-related information may appear in the slice header without appearing in the picture header.

[0245] picture header: If pps_cca_info_in_ph_flag is 1, it indicates that CCA-related information appears in the picture header, and the decoder can read the CCA filtering coefficients based on the cca_structure() structure in the picture header, and the syntax semantics are shown in Table 3 below.

[0246] [Table 3]

[0247] If pps_cca_info_in_ph_flag is 0, it indicates that CCA-related information may appear in the slice header without appearing in the picture header. A decoder can read the CCA filtering coefficients based on the cca_structure() structure in the slice header, and the syntax semantics are shown in Table 4 below.

[0248] [Table 4]

[0249] As shown in Table 5 below, the decoder determines whether the current image or slice uses CCA based on the syntax elements in the cca_structure() structure, and if CCA is used, further obtains the filtering coefficients of the luma component or chroma component to complete the CCA operation.

[0250] [Table 5]

[0251] When cca_enabled_flag is 1, it indicates that CCA is used, and when it is 0, it indicates that CCA is not used.

[0252] When cca_luma_enabled_flag is 1, it indicates that the luma component of the current slice uses CCA, and when it is 0, it indicates that the luma component of the current slice does not use CCA.

[0253] num_cca_luma_coeff indicates the number of luma filter coefficients. It is usually a predetermined value and does not need to be transmitted. Different embodiments may choose different numbers of filters.

[0254] cca_luma_coeff_abs[i] indicates the absolute value of the i-th coefficient of the current slice luminance.

[0255] cca_luma_coeff_signed[i] indicates the i-th coefficient symbol of the current slice luminance.

[0256] When cca_cb_enabled_flag is 1, it indicates that the Cb component of the current slice uses CCA, and when it is 0, it indicates that the Cb component of the current slice does not use CCA.

[0257] When cca_cr_enabled_flag is 1, it indicates that the Cr component of the current slice uses CCA, and when it is 0, it indicates that the Cr component of the current slice does not use CCA.

[0258] cca_cb_coeff_abs[i] indicates the i-th coefficient absolute value of the Cb component of the current slice.

[0259] cca_cb_coeff_signed[i] indicates the i-th coefficient symbol of the Cb component of the current slice.

[0260] cca_cr_coeff_abs[i] indicates the i-th coefficient absolute value of the Cr component of the current slice.

[0261] cca_cr_coeff_signed[i] indicates the i-th coefficient symbol of the Cr component of the current slice.

[0262] num_cca_chroma_coeff indicates the number of chroma filter coefficients. It is usually a predetermined value and does not need to be transmitted. Different embodiments may choose a different number of filters.

[0263] The syntax structure used for the luminance component only is shown in Table 6 below.

[0264] [Table 6]

[0265] The syntax structure used for chroma components only is shown in Table 7 below.

[0266] [Table 7]

[0267] Block Level Switch: When setting a block-level switch for CCA operation and controlling the CCA switch operation of each block, the decoder needs to analyze the block-level switch as shown in Table 8 below.

[0268] [Table 8]

[0269] cca_block_flag[i] identifies whether the block uses CCA, with 1 indicating yes and 0 indicating no. i is 0 for the luminance component, 1 for the chroma Cb component, and 2 for the chroma Cr component.

[0270] Block-level CCA coefficient transmission: When CCA is performed at the block level, it is necessary to transmit the CCA coefficients at the block level.

[0271] The brightness is shown in Table 9 below.

[0272] [Table 9]

[0273] The chromas are shown in Table 10 below.

[0274] [Table 10]

[0275] Coefficient Fusion Mode: If the block level coefficients can be merged with the upper and left adjacent blocks, a merge switch needs to be transmitted.

[0276] The brightness is shown in Table 11 below.

[0277] [Table 11]

[0278] cca_block_merge_flag[i][Blk AddrX][Blk AddrY] identifies whether the block uses CCA merge mode, with 1 indicating yes and 0 indicating no. i is 0 for the luminance component, 1 for the chroma Cb component, and 2 for the chroma Cr component.

[0279] cca_block_merge_mode_flag[0][Blk AddrX][Blk AddrY] identifies the CCA merge mode type of the block, where 1 indicates that the CCA coefficients of the block adjacent above are used, and 0 indicates that the CCA coefficients of the block adjacent to the left are used. i indicates the luminance component when 0, the chroma Cb component when 1, and the chroma Cr component when 2.

[0280] The chromas are shown in Table 12 below.

[0281] [Table 12]

[0282] The Cross Components CCA modes are shown in Table 13 below.

[0283] [Table 13]

[0284] cc_cca_structure() The calling locations of cc_cca_structure() and cca_structure() are the same, as shown in Table 14 below.

[0285] [Table 14]

[0286] When cc_cca_enabled_flag is 1, it indicates that CC-CCA is used, and when it is 0, it indicates that CC-CCA is not used. If this syntax element is not present, it indicates that cca_enabled_flag is 0.

[0287] When cc_cca_cb_enabled_flag is 1, it indicates that the Cb component of the current slice uses CC-CCA, and when it is 0, it indicates that the Cb component of the current slice does not use CC-CCA.

[0288] cc_cca_cr_enabled_flag indicates that the Cr component of the current slice uses CC-CCA when it is 1, and indicates that the Cr component of the current slice does not use CC-CCA when it is 0.

[0289] cc_cca_cb_coeff_abs[i] indicates the i-th coefficient absolute value of the Cb component of the current slice.

[0290] cc_cca_cb_coeff_signed[i] indicates the i-th coefficient symbol of the Cb component of the current slice.

[0291] cc_cca_cr_coeff_abs[i] indicates the i-th coefficient absolute value of the Cr component of the current slice.

[0292] cc_cca_cr_coeff_signed[i] indicates the i-th coefficient symbol of the Cr component of the current slice.

[0293] num_cc_cca_chroma_coeff indicates the number of chroma filter coefficients. It is usually a predetermined value and does not need to be transmitted. Different embodiments may choose a different number of filters.

[0294] According to another aspect of an embodiment of the present disclosure, there is further provided a video encoding / decoding loop filtering device. FIG. 15 is a block diagram of a loop filtering device according to an embodiment of the present disclosure. As shown in FIG. 15 , the device includes: a first filtering module 152 configured to filter the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; a second filtering module 154 configured to filter the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; a first determination module 156 configured to determine filter coefficients based on neighborhood information included in convolution templates of the first reconstructed video unit and the second reconstructed video unit, the neighborhood information being information about neighboring pixel points centered on a pixel point in the convolution template; and a third filtering module 158 configured to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain a third reconstructed video unit.

[0295] In one embodiment, the first determination module 156 is further configured to: for each pixel point in the video unit of the reconstructed image and the video unit of the original image, respectively determine a difference value between the pixel point and a pixel point at each neighboring position in the convolution template; multiply the difference values ​​respectively to obtain a first pixel product; determine a difference value between the pixel point in the video unit of the original image and the pixel point in the video unit of the reconstructed image to obtain a covariance; and determine the filter coefficients based on the covariance based on a Wiener filtering method.

[0296] In one embodiment, the video unit of the reconstructed image comprises a reconstructed image, or a slice, or a coding block, or a coding tree block.

[0297] In one embodiment, the third filtering module 158: performing a filtering process on luminance components of the first reconstructed video unit and the second reconstructed video unit based on the filter coefficients according to the convolution template; performing a filtering process on a first chrominance component of the first reconstructed video unit and a first chrominance component of the second reconstructed video unit based on the filter coefficients according to the convolution template; and performing a filtering operation on the first reconstructed video unit and a second chrominance component of the second reconstructed video unit based on the filter coefficients based on the convolution template.

[0298] In one embodiment, the first filtering module 152 is further configured to perform a deblocking filter DBF on the reconstructed image, or to perform a sample adaptive offset SAO on the reconstructed image, or to perform a deblocking filter DBF and a sample adaptive offset SAO on the reconstructed image, or to perform a luminance mapping LMCS process based on chroma scaling on the reconstructed image.

[0299] In one embodiment, the second filtering module 154 is further configured to: perform neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or perform neural network loop filtering on the reconstructed image and perform a weighted addition of the neural network loop filtering result and the first reconstructed video unit to obtain the second reconstructed video unit; or perform neural network loop filtering on the reconstructed image and perform intensity adjustment on the neural network loop filtering result based on an original image to obtain the second reconstructed video unit; or perform other loop filtering other than the neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of a deblocking filter DBF, a sample adaptive offset SAO, and a luminance mapping based on chroma scaling LMCS.

[0300] In one embodiment, the device comprises: a second determination module configured to determine a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image, and to determine a second loss between pixels of a video unit of the reconstructed image before filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of the original image; and a CCA filtering module configured to determine whether to perform CCA filtering on a video unit of the current image based on the first loss and the second loss.

[0301] In one embodiment, the device comprises: a third determination module configured to determine a first loss between pixels of the video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image, filter with a predetermined filter coefficient, and determine a third loss between pixels of the video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of the original image; and a fourth filtering module configured to determine, based on the first loss and the third loss, to perform the filtering process using the filter coefficients or to determine to perform the filtering process using predetermined filter coefficients.

[0302] In one embodiment, the device comprises: a fourth determination module configured to determine a first loss between pixels of the video unit of the reconstructed image after filtering and pixels of an original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; determine a second loss between pixels of the video unit of the reconstructed image before filtering and pixels of the original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; and determine a fourth loss between pixels of the video unit of the reconstructed image after filtering and pixels of the original image by performing the filtering on the video unit of the current image based on filter coefficients of neighboring video units of the video unit of the current image; and a fifth filtering module configured to determine filter coefficients to be used for the video unit of the current image based on the first loss, the second loss, and the fourth loss, and to perform filtering based on the determined filter coefficients.

[0303] In one embodiment, the device comprises: a fifth determination module configured to determine a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image, and to determine a second loss between pixels of a video unit of the reconstructed image before filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image; and a confirmation module configured to determine whether to perform the filtering on the video unit of the current image based on the first loss and the second loss, recalculate filter coefficients for the filtered video unit, and again determine whether to perform filtering based on the first loss and the second loss, and after repeating N times, confirm whether to perform the filtering on the video unit of the current image.

[0304] In one embodiment, the third filtering module 158 is further configured to calculate a difference value between the central pixel point and a neighboring pixel point of the convolution template based on the first reconstructed video unit, multiply by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a first result, and simultaneously calculate a difference value between the central pixel point and a neighboring pixel point of the convolution template based on the second reconstructed video unit, multiply by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a second result, add the first result, the second result and the second reconstructed video unit, and perform rounding to obtain the third reconstructed video unit.

[0305] In one embodiment, the convolution template includes one of a diamond template, a square template, and a cross template.

[0306] According to another aspect of the embodiment of the present disclosure, there is further provided a video decoding device, and FIG. 16 is a block diagram of the video decoding device according to the embodiment of the present disclosure. As shown in FIG. 16, the device includes: an acquisition module 162 configured to acquire a bitstream including an encoded video sequence to acquire loop filtering control information corresponding to a loop filtering scheme; a first filtering module 152 configured to filter the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; a second filtering module 154 configured to filter the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; and a third filtering module 158 configured to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain a third reconstructed video unit.

[0307] In one embodiment, the first filtering module 152 is further configured to perform a deblocking filter DBF on the reconstructed image, or to perform a sample adaptive offset SAO on the reconstructed image, or to perform a deblocking filter DBF and a sample adaptive offset SAO on the reconstructed image, or to perform a luminance mapping LMCS process based on chroma scaling on the reconstructed image.

[0308] In one embodiment, the second filtering module 154 is further configured to: perform neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or perform neural network loop filtering on the reconstructed image and perform a weighted addition of the neural network loop filtering result and the first reconstructed video unit to obtain the second reconstructed video unit; or perform neural network loop filtering on the reconstructed image and perform intensity adjustment on the neural network loop filtering result based on an original image to obtain the second reconstructed video unit; or perform other loop filtering other than neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit, wherein the other loop filtering includes at least one of a deblocking filter DBF, a sample adaptive offset SAO, and a luminance mapping LMCS based on chroma scaling.

[0309] In one embodiment, the third filtering module 158 is further configured to calculate a difference value between the central pixel point and a neighboring pixel point of the convolution template based on the first reconstructed video unit, multiply by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a first result, and simultaneously calculate a difference value between the central pixel point and a neighboring pixel point of the convolution template based on the second reconstructed video unit, multiply by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a second result, add the first result, the second result and the second reconstructed video unit, and perform rounding to obtain the third reconstructed video unit.

[0310] In one embodiment, the loop filtering information is a sequence-level filtering switch identifier indicating whether to turn on / off the loop filtering scheme for the video sequence to which the current video frame belongs; a filtering picture level identifier indicating that the loop filtering model information appears in picture header information or slice header information; and loop filtering model structure information that instructs the loop filtering to be performed on a video unit to which the loop filtering method is applied.

[0311] In one embodiment, the device comprises: and a sixth determination module configured to determine a value of the sequence level filtering switch identifier, and to determine to turn on the loop filtering for the current encoded video sequence if the value of the sequence level filtering switch identifier is a first value, and to determine to turn off the loop filtering for the current encoded video sequence if the value of the sequence level filtering switch identifier is a second value, wherein the first value is different from the second value.

[0312] In one embodiment, the device comprises: The seventh determination module is configured to determine a value of the filtering picture level identifier, and if the value of the filtering picture level identifier is a third value, determine that the loop filtering model information appears in a picture header, and if the value of the filtering picture level identifier is a fourth value, determine that the loop filtering model information appears in a slice header, wherein the third value is different from the fourth value.

[0313] In one embodiment, the video unit , image or slice Includes. In one embodiment, the loop filtering model structure information is a video unit filtering switch identifier for instructing a video unit to turn on / off the loop filtering scheme; a video unit component switch identifier for instructing a video unit component to turn on / off the loop filtering scheme; a video unit block level filtering switch identifier for instructing whether to turn on / off the loop filtering scheme for each level in the video unit; a video unit filtering coefficient absolute value indicating a filtering coefficient absolute value corresponding to each level in the video unit; and a video unit filtering coefficient symbol indicating a filtering coefficient symbol corresponding to each level in the video unit; The video unit component is the luminance component of the video unit, The first chroma component of the video unit, and the second chroma component of the video unit.

[0314] In one embodiment, the device comprises: and an eighth determination module configured to determine a value of the video unit filtering switch identifier and, based on the value, determine whether to turn on the loop filtering or turn off the loop filtering for a current video unit.

[0315] In one embodiment, the device comprises: The video unit further includes a ninth determination module configured to: determine a value of the video unit luma component switch identifier, and determine, based on the value, whether to turn on the loop filtering for a current video unit luma component or turn off the loop filtering for a current video unit luma component; determine a value of a first chroma component switch identifier of the video unit, and determine, based on the value, whether to turn on the loop filtering for the first chroma component of the current video unit or turn off the loop filtering for the first chroma component of the current video unit; determine a value of a second chroma component switch identifier of the video unit, and determine, based on the value, whether to turn on the loop filtering for a second chroma component of the current video unit or turn off the loop filtering for the second chroma component of the current video unit.

[0316] In one embodiment, the device comprises: and a tenth decision module configured to determine a value of the video unit block level filtering switch identifier and, based on the value, determine whether to turn on the loop filtering for coding blocks in the video unit or turn off the loop filtering for coding blocks in the video unit.

[0317] In one embodiment, the device comprises: The sixth filtering module is configured to filter the video unit that turns on the loop filtering scheme in the loop filtering scheme based on the video unit filtering coefficient absolute value and the video unit filtering coefficient symbol.

[0318] In one embodiment, the loop filtering model structure information is a video unit block level filter coefficient fusion mode switch identifier indicating whether to turn on coefficient fusion mode for the coding blocks and components of the coding blocks in the video unit; a video unit block level filter coefficient fusion mode type identifier indicating a position of a neighboring coding block of a current coding block in a video unit, the current coding block performing the loop filtering using filter coefficients of the neighboring coding block; The coded block components include at least one of a luminance component, a first chroma component, and a second chroma component.

[0319] In one embodiment, the loop filtering model structure information further includes a cross-component filtering mode switch identifier indicating whether to turn on cross-component filtering mode for a coding block and a component of the coding block in a video unit.

[0320] In an embodiment of the present disclosure, there is further provided a computer-readable storage medium having a computer program stored thereon, the computer program being configured to perform the steps of any of the method embodiments described above when executed.

[0321] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB disk, a read-only memory (abbreviated as ROM), a random access memory (abbreviated as RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0322] In an embodiment of the present disclosure, there is further provided an electronic device including a memory having a computer program stored therein, and a processor configured to execute the computer program to implement the steps of any of the above method embodiments.

[0323] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, the transmission device being connected to the processor and the input / output device being connected to the processor.

[0324] For specific examples in this embodiment, reference can be made to the examples described in the above embodiments and exemplary embodiments, so repeated explanations of this embodiment will be omitted here.

[0325] Obviously, those skilled in the art should understand that each module or step in the present disclosure may be implemented in a general-purpose computing device, may be centralized in a single computing device, or may be distributed across a network of multiple computing devices, may be implemented in program code executable by a computing device, may be stored in a storage device and executed on a computing device, and may in some cases execute the steps illustrated or described in the specification in a different order than in the specification, or may be implemented by forming each module into an integrated circuit module, or may implement multiple modules or steps into a single integrated circuit. Thus, the present disclosure is not limited to any specific combination of hardware and software.

[0326] The above is only a preferred embodiment of the present disclosure, and does not limit the present disclosure, and those skilled in the art can make various modifications and changes to the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present disclosure should be included within the protection scope of the present disclosure.

Claims

1. filtering the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; filtering the reconstructed image with a second filter to obtain a second reconstructed video unit; determining filter coefficients based on neighborhood information included in convolution templates of the first reconstructed video unit and the second reconstructed video unit, the neighborhood information being information about neighboring pixel points centered around a pixel point in the convolution template; filtering the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain a third reconstructed video unit.

2. The step of determining filter coefficients based on neighborhood information included in convolution templates of the first reconstructed video unit and the second reconstructed video unit includes: For each pixel point in the video unit of the reconstructed image and the video unit of the original image, determine a difference value between each pixel point and a pixel point at each neighboring position in the convolution template, and multiply the difference values ​​to obtain a first pixel product; determine a difference value between the pixel point in the video unit of the original image and the pixel point in the video unit of the reconstructed image to obtain a covariance; and determining the filter coefficients based on the covariance based on a Wiener filtering method.

3. The method of claim 2 , wherein the video units of the reconstructed image comprise reconstruction processed images, or slices, or coding blocks, or coding tree blocks.

4. filtering the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain a third reconstructed video unit, performing a filtering process on luminance components of the first reconstructed video unit and the second reconstructed video unit based on the filter coefficients according to the convolution template; performing a filtering process on a first chrominance component of the first reconstructed video unit and a first chrominance component of the second reconstructed video unit based on the filter coefficients, based on the convolution template; and performing a filtering process on the first reconstructed video unit and a second chrominance component of the second reconstructed video unit based on the filter coefficients based on the convolution template.

5. The step of filtering the reconstructed image of the current image with a first filter includes: performing a deblocking filter (DBF) on the reconstructed image; or performing sample adaptive offset (SAO) on the reconstructed image; or performing a deblocking filter (DBF) and a sample adaptive offset (SAO) on the reconstructed image; or The method of claim 1 , further comprising: performing a luminance mapping LMCS process based on chroma scaling on the reconstructed image.

6. filtering the reconstructed image with a second filter to obtain a second reconstructed video unit; performing neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image, and performing weighted addition of the neural network loop filtering result and the first reconstructed video unit to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image, and performing intensity adjustment on the neural network loop filtering result based on the original image to obtain the second reconstructed video unit; or 2. The method of claim 1, comprising: performing other loop filtering on the reconstructed image other than the neural network loop filtering to obtain the second reconstructed video unit, the other loop filtering including at least one of a deblocking filter (DBF), a sample adaptive offset (SAO), and a luma mapping based on chroma scaling (LMCS).

7. determining a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image; determining a second loss between pixels of a video unit of the reconstructed image before filtering and pixels of the original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; The method of claim 1 , further comprising: determining whether to perform CCA filtering on the video unit of the current image based on the first loss and the second loss.

8. determining a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image; filtering with predetermined filter coefficients, and determining a third loss between pixels of the video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of the original image; The method of claim 1 , further comprising: determining, based on the first loss and the third loss, to perform the filtering process using the filter coefficients or to perform the filtering process using predetermined filter coefficients.

9. determining a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image; determining a second loss between pixels of a video unit of the reconstructed image before filtering and pixels of an original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; performing the filtering on the video unit of the current image based on filter coefficients of neighboring video units of the video unit of the current image to determine a fourth loss between pixels of the video unit of the reconstructed image after filtering and pixels of the original image; 2. The method of claim 1 , further comprising: determining filter coefficients to be used for a video unit of the current image based on the first loss, the second loss, and the fourth loss; and performing a filtering process based on the determined filter coefficients.

10. determining a first loss between pixels of a video unit of the reconstructed image after filtering based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients and pixels of an original image; determining a second loss between pixels of a video unit of the reconstructed image before filtering and pixels of an original image based on the first reconstructed video unit, the second reconstructed video unit, and the filter coefficients; 2. The method of claim 1, further comprising: determining whether to perform the filtering on the video unit of the current image based on the first loss and the second loss; recalculating filter coefficients for the filtered video unit; and determining whether to perform the filtering on the video unit of the current image based on the first loss and the second loss again; and repeating this N times, confirming whether to perform the filtering on the video unit of the current image.

11. filtering the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain a third reconstructed video unit, Calculating a difference value between a central pixel point and a neighboring pixel point of the convolution template according to the first reconstructed video unit, and multiplying the difference value by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a first result; and simultaneously calculating a difference value between a central pixel point and a neighboring pixel point of the convolution template according to the second reconstructed video unit, and multiplying the difference value by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a second result; adding the first result, the second result, and the second reconstructed video unit and performing rounding to obtain the third reconstructed video unit.

12. The method of any one of claims 1 to 11, wherein the convolution template comprises one of a diamond template, a square template, and a cross template.

13. obtaining a bitstream including an encoded video sequence and obtaining loop filtering control information corresponding to a loop filtering scheme, the loop filtering control information including filter coefficients; filtering the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; filtering the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; and filtering the first reconstructed video unit and the second reconstructed video unit based on convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain a third reconstructed video unit.

14. The step of filtering the reconstructed image of the current image with a first filter comprises: performing a deblocking filter (DBF) on the reconstructed image; or performing sample adaptive offset (SAO) on the reconstructed image; or performing a deblocking filter (DBF) and a sample adaptive offset (SAO) on the reconstructed image; or The method of claim 13 , comprising performing a chroma scaling based luminance mapping LMCS process on the reconstructed image.

15. filtering the reconstructed image of the current image with a second filter to obtain the second reconstructed video unit; performing neural network loop filtering on the reconstructed image to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image, and performing weighted addition of the neural network loop filtering result and the first reconstructed video unit to obtain the second reconstructed video unit; or performing neural network loop filtering on the reconstructed image, and performing intensity adjustment on the neural network loop filtering result based on the original image to obtain the second reconstructed video unit; or 14. The method of claim 13, comprising: performing other loop filtering on the reconstructed image other than neural network loop filtering to obtain the second reconstructed video unit, the other loop filtering including at least one of a deblocking filter (DBF), a sample adaptive offset (SAO), and a luma mapping based on chroma scaling (LMCS).

16. The step of filtering the first reconstructed video unit and the second reconstructed video unit based on the convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain a third reconstructed video unit includes: Calculating a difference value between a central pixel point and a neighboring pixel point of the convolution template according to the first reconstructed video unit, and multiplying the difference value by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a first result; and simultaneously calculating a difference value between a central pixel point and a neighboring pixel point of the convolution template according to the second reconstructed video unit, and multiplying the difference value by a filter coefficient at a position corresponding to the neighboring pixel point to obtain a second result; adding the first result, the second result, and the second reconstructed video unit and performing rounding to obtain the third reconstructed video unit.

17. The loop filtering information is a sequence-level filtering switch identifier indicating whether to turn on / off the loop filtering scheme for the video sequence to which the current video frame belongs; a filtering picture level identifier indicating that the loop filtering model information appears in picture header information or slice header information; and loop filtering model structure information that instructs the loop filtering to be performed on a video unit to which the loop filtering scheme is applied.

18. determining a value for the sequence level filtering switch identifier; 18. The method of claim 17, further comprising: determining to turn on the loop filtering for a current encoded video sequence if a value of the sequence level filtering switch identifier is a first value; and determining to turn off the loop filtering for a current encoded video sequence if a value of the sequence level filtering switch identifier is a second value, the first value being different from the second value.

19. determining a value for the filtering picture level identifier; 18. The method of claim 17, further comprising: determining that the loop filtering model information appears in a picture header if the value of the filtering picture level identifier is a third value; and determining that the loop filtering model information appears in a slice header if the value of the filtering picture level identifier is a fourth value, wherein the third value is different from the fourth value.

20. The method of claim 17 , wherein the video unit comprises an image or a video slice in a video.

21. The loop filtering model structure information is a video unit filtering switch identifier for instructing a video unit to turn on / off the loop filtering scheme; a video unit component switch identifier for instructing a video unit component to turn on / off the loop filtering scheme; a video unit block level filtering switch identifier for instructing whether to turn on / off the loop filtering scheme for each level in the video unit; a video unit filtering coefficient absolute value indicating a filtering coefficient absolute value corresponding to each level in the video unit; and a video unit filtering coefficient symbol indicating a filtering coefficient symbol corresponding to each level in the video unit; The video unit component is the luminance component of the video unit, the first chroma component of the video unit, and 20. The method of claim 17, wherein the second chrominance component of the video unit is a second chrominance component of the video unit.

22. 22. The method of claim 21, further comprising determining a value of the video unit filtering switch identifier and determining, based on the value, whether to turn the loop filtering on or turn the loop filtering off for a current video unit.

23. determining a value of the video unit luma component switch identifier, and determining based on the value whether to turn on the loop filtering for a current video unit luma component or to turn off the loop filtering for a current video unit luma component; determining a value of a first chroma component switch identifier of the video unit, and determining, based on the value, whether to turn on the loop filtering for a first chroma component of the current video unit or turn off the loop filtering for the first chroma component of the current video unit; 22. The method of claim 21 , further comprising: determining a value of a second chroma component switch identifier of the video unit; and determining, based on the value, whether to turn on the loop filtering for the second chroma component of the current video unit or turn off the loop filtering for the second chroma component of the current video unit.

24. 22. The method of claim 21 , further comprising determining a value of the video unit block level filtering switch identifier and determining, based on the value, whether to turn on the loop filtering for coding blocks in a video unit or turn off the loop filtering for coding blocks in a video unit.

25. 14. The method of claim 13, further comprising: filtering the video unit that turns on the loop filtering scheme with the loop filtering scheme based on the video unit filtering coefficient absolute value and the video unit filtering coefficient symbol.

26. The loop filtering model structure information is a video unit block level filter coefficient fusion mode switch identifier indicating whether to turn on coefficient fusion mode for the coding blocks and components of the coding blocks in the video unit; a video unit block level filter coefficient fusion mode type identifier indicating a position of a neighboring coding block of a current coding block in a video unit, the current coding block performing the loop filtering using filter coefficients of the neighboring coding block; 20. The method of claim 17, wherein the coded block components include at least one of a luma component, a first chroma component, and a second chroma component.

27. The loop filtering model structure information is 18. The method of claim 17, further comprising a cross-component filtering mode switch identifier that indicates whether cross-component filtering mode is on for a coding block and a component of the coding block in a video unit.

28. a first filtering module configured to filter the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; a second filtering module configured to filter the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; a first determination module configured to determine filter coefficients based on neighborhood information included in convolution templates of the first reconstructed video unit and the second reconstructed video unit, wherein the neighborhood information is information about neighboring pixel points centered on a pixel point in the convolution template; and a third filtering module configured to filter the first reconstructed video unit and the second reconstructed video unit based on the convolution template and the filter coefficients to obtain a third reconstructed video unit.

29. an acquisition module configured to acquire a bitstream including an encoded video sequence to acquire loop filtering control information corresponding to a loop filtering scheme; a first filtering module configured to filter the reconstructed image of the current image with a first filter to obtain a first reconstructed video unit; a second filtering module configured to filter the reconstructed image of the current image with a second filter to obtain a second reconstructed video unit; a third filtering module configured to filter the first reconstructed video unit and the second reconstructed video unit based on convolution templates of the first reconstructed video unit and the second reconstructed video unit and the filter coefficients to obtain a third reconstructed video unit.

30. A computer-readable storage medium having stored thereon a computer program, the computer program being configured to, when executed, perform the method of any one of claims 1 to 12 and claims 13 to 27.

31. An electronic device comprising: a memory in which a computer program is stored; and a processor configured to execute the computer program to perform the method of any one of claims 1 to 12 and claims 13 to 27.

Images